Free jis software for linux

ShiftJIS::Regexp contains regular expressions in Shift-JIS.

SYNOPSIS

use ShiftJIS::Regexp qw(:all);

match($string, p{Hiragana}{2}p{Digit}{2});
match($string, pH{2}pD{2});
# these two are equivalent:

This module provides some functions to use regular expressions in Shift-JIS on the byte-oriented perl.

The legal Shift-JIS character in this module must match the following regular expression:

[x00-x7FxA1-xDF]|[x81-x9FxE0-xFC][x40-x7Ex80-xFC]

To avoid false matching in multibyte encoding, this module uses anchoring technique to ensure each matching position places at the character boundaries.

cf. perlfaq6, "How can I match strings with multibyte characters?"

Functions

re(PATTERN)

re(PATTERN, MODIFIER)

Returns a regular expression parsable by the byte-oriented perl.

PATTERN is specified as a string. MODIFIER is specified as a string. Modifiers in the following list are allowed.

i case-insensitive pattern (only for ascii alphabets)
I case-insensitive pattern (greek, cyrillic, fullwidth latin)
j hiragana-katakana-insensitive pattern (but halfwidth katakana
are not considered.)

s treat string as single line
m treat string as multiple lines
x ignore whitespace (i.e. [x20nrtf]) unless backslashed
or inside a character class; but comments are not recognized!

o once parsed (not compiled!) and the result is cached internally.
o modifier
while ( ) {
print replace($_, (perl), $1, igo);
}
is more efficient than

while ( ) {
print replace($_, (perl), $1, ig);
}

because in the latter case the pattern is parsed every time
whenever the function is called.

match(STRING, PATTERN)

match(STRING, PATTERN, MODIFIER)

An emulation of m// operator aware of Shift-JIS. But, to emulate @list = $string =~ m/PATTERN/g, the pattern should be parenthesized (capturing parentheses are not added automatically).

@list = match($string, pH, g); # wrong; returns garbage!
@list = match($string,(pH),g); # good
PATTERN is specified as a string. MODIFIER is specified as a string.
i,I,j,s,m,x,o please see re().

g match globally
z tell the function the pattern matches an empty string
(sorry, due to the poor auto-detection)

replace(STRING or SCALAR REF, PATTERN, REPLACEMENT)

replace(STRING or SCALAR REF, PATTERN, REPLACEMENT, MODIFIER)

An emulation of s/// operator but aware of Shift-JIS.
If a reference to a scalar is specified as the first argument, substitutes the referent scalar and returns the number of substitutions made. If a string (not a reference) is specified as the first argument, returns the substituted string and the specified string is unaffected.

MODIFIER is specified as a string.

i,I,j,s,m,x,o please see re().
g,z please see match().

jsplit(PATTERN or ARRAY REF of [PATTERN, MODIFIER], STRING)

jsplit(PATTERN or ARRAY REF of [PATTERN, MODIFIER], STRING, LIMIT)

An emulation of CORE::split but aware of Shift-JIS.
In scalar/void context, it does not split into the @_ array; in scalar context, only returns the number of fields found.
PATTERN is specified as a string. But as PATTERN has no special meaning; it splits the string on a single space similarly to CORE::split / /.

When you want to split the string on whitespace, pass an undefined value as PATTERN or use the splitspace() function.

jsplit(undef, " x81x40 This is x81x40 perl.");
splitspace(" x81x40 This is x81x40 perl.");
# (This, is, perl.)

If you want to pass pattern with modifiers, specify an arrayref of [PATTERN, MODIFIER] as the first argument. You can also use "Embedded Modifiers").

MODIFIER is specified as a string.

i,I,j,s,m,x,o please see re().

splitspace(STRING)

splitspace(STRING, LIMIT)

This function emulates CORE::split( , STRING, LIMIT). It returns a list given by split STRING on whitespace including "x81x40" (IDEOGRAPHIC SPACE). Leading whitespace characters do not produce any field.

Note: splitspace(STRING, LIMIT) is equivalent to jsplit(undef, STRING, LIMIT).

splitchar(STRING)

splitchar(STRING, LIMIT)

This function emulates CORE::split(//, STRING, LIMIT). It returns a list given by split of STRING into characters.
Note: splitchar(STRING, LIMIT) is equivalent to jsplit(, STRING, LIMIT).

Jcode is a Japanese Charset Handler.

SYNOPSIS

use Jcode;
#
# traditional
Jcode::convert($str, $ocode, $icode, "z");
# or OOP!
print Jcode->new($str)->h2z->tr($from, $to)->utf8;

< Japanese document is now available as Jcode::Nihongo. >

Jcode.pm supports both object and traditional approach. With object approach, you can go like;

$iso_2022_jp = Jcode->new($str)->h2z->jis;

Which is more elegant than:

$iso_2022_jp = $str;
&jcode::convert($iso_2022_jp, jis, &jcode::getcode($str), "z");

For those unfamiliar with objects, Jcode.pm still supports getcode() and convert().

If the perl version is 5.8.1, Jcode acts as a wrapper to Encode, the standard charset handler module for Perl 5.8 or later.

Bit-mapped Japanese font parser is a font parser. Note, this package doesnt include the actual font data. To get the font data you need to download it from the download section in the left.
Then move *.jfr into the directory where you unpacked this parser, and follow with the quick instructions.
Quick instructions:
Complete parse requires about 4 megabytes of free disk space. This is a huge improvement over the original version which required almost 45 megabytes.
1. make
2. make parse
3. watch the progress indicator
4. mv *.pcf.gz /usr/X11R/lib/X11R6/fonts/misc
5. make clean
6. HUP your font server if you use one
7. xset fp rehash
8. xlsfonts | grep kanji
/usr/X11R/lib/X11R6/fonts/misc is the standard location for all sorts of random bit-mapped fonts, but you might have a special location. Substitute that in step 4.
About:
I came across a number of these "raster fonts" a while ago. Quick look inside the files proved that they are bit-mapped fonts, and the format looked pretty straight-forward. I wrote the original parser for these just guessing the values, basically by experimenting and playing around. Later on I came across some docs on the subject - looks like these fonts were used in Windows 3.1 Japanese edition to substitute back-then low quality Japanese TTF fonts at small point sizes. These were designed using full-scale 16 bit programming techniques.
Quick info about the font format, there are some headers, then follows a "segment table" which is basically a table with pointers inside the font file where to locate a particular chunk of data. Because the 16 bit way of accessing memory is by using 65k "segments", each file is virtually split into < 65k segments which get loaded into separate memory areas, and then there is a algorithm how to assemble whatever character by using the segment number and offset. Anyway, with 32 bit access all of that doesnt really matter. In my implementation I just mmap the whole file and read it all out of memory.
Generating table.h was a LOT of work! First, I took the codearea table out of one of the jfr files (this maps shift-jis code to the character number inside the font file), and extracted the number ranges. These were shift-jis, of course, and X uses jis0208. There is no converter from a shift
jis byte into jis0208. So I had to write one. Taking iconv, and some tables from glibc 2.1.93, I hacked together something which converted the shift-jis data into ucs4 (unicode, I guess) and then from that into jis0208. The code to the converter is about 500k thanks to the huge jis->unicode->jis conversion tables, and you wont need it unless you get a jfr font with a different encoding table (unlikely). Anyway. After I got the font format figured out and converted the character table, everything else was pretty easy. Note some bit hackery in the bitmapXX() functions which was necessary to present the font data in a usable format. Also notice cool use of function pointers to select a conversion function at runtime.
Enhancements:
- This version uses correct JISX0208 tables, and is much faster.

iconvircproxy project is a proxy for IRC users who want to talk in different character set environments.

In an ideal world, everyone would use UTF-8 for text transmission. UTF-8 supports nearly all characters of all languages, including the thousands of ideographs from Chinese.

However, in the world we live in, people use different encodings. Finnish people use ISO-8859-1, ISO-8859-15 or Windows-1252; Japanese people use ISO-2022-JP, SHIFT-JIS or EUC-JP; Polish people use ISO-8859-2, and so on. People using different encodings will see other peoples text as an unreadable mess of code.

This proxy is a solution for IRC users who want to be able to talk with everyone despite of their character set differences.

The primary audience of this program are the European IRC users who occassionally talk on Japanese channels. With the proxy, all you need is an IRC client that understands and speaks fluently UTF-8.

The proxy supports all the ISO-8859 family character sets, ISO-2022-JP, SHIFT-JIS Japanese encodings and the UTF-8 unicode encoding, and will transparently use them with the "in Rome, do like the Romans do" principle.

Note: As of version 1.4.2, UTF-8 is not the only supported client encoding, but it is still the only recommended one.

Gauche is an R5RS Scheme implementation developed to be a handy script interpreter, which allows programmers and system administrators to write small to large scripts for their daily chores. Quick startup, built-in system interface, native multilingual support are some of my goals.
Gauche application runs on several Unix-like platforms.
Main features:
- Covers R5RS, almost.
- R5RS macro is supported as all of define-syntax, let-syntax, letrec-syntax forms. I think the speed of macro expansion is important as a script interpreter, and wrote R5RS macro expander in C, integrated in the internal compiler engine.
- Numeric operations are supported on fixnum, bignum, flonum and complex.
- Supports the following SRFIs.
- Module system:a simple module system, API compatible to STklos
- Object system:CLOS-like object system with metaobject protocol. Almost API compatible to STklos . It is also similar to Guile s object system.
- Multibyte string support: Strings are represented by multibyte string internally. You can use UTF-8, EUC-JP, Shift-JIS or no multibyte encoding by configure-time choice. Conversion between native coding system and external coding system is supported by port objects.
- Multibyte regexp: Regular expression matcher is aware of multibyte string; you can use multibyte characters both in patterns and matched strings.
- Reader extension: literal regexp and char-set, string interpolation: Extended reader recognizes #/.../ as an regular expression, and #[...] as a character set. Handy to write one-liners. (e.g. (rxmatch-substring (rxmatch #/(d+)/ "abc123def")) ==> "123")
- Also "string interpolation" is supported (e.g. (display #`"1 + 2 = ,(+ 1 2)n")).
- System interface: Covers most of POSIX.1 and some other features common in Unices. See section 6.19 of the reference manual for details.
- Network interface: Has API for socket-based network interface, including IPv6 if the OS suppots it.
- Multithreading: On some platforms, multithreading is supported on top of pthreads. Scheme-level API conforms SRFI-18.
- DBM interface: Interface to DBM-like libraries (dbm, ndbm and/or gdbm) if the system provides them.
- XML parsing: Oleg Kiselyovs SXML tools are included.
- OpenGL binding: OpenGL binding is provided in a separate package.
- GTK binding: GTK2 binding is provided in a separate package
Instalation
% gzcat Gauche-0.8.3.tgz | tar xf -
% cd Gauche-0.8.3
% ./configure
% make
% make install
Enhancements:
- Bug fixes include numerous edge-case bugs that caused SEGV or numeric error were fixed.
- Now pthread support is automatically enabled on suitable platforms even if you dont give --enable-threads configure option.