Free unicode character maps software for linux

Unicode::Map8 is a mapping table between 8-bit chars and Unicode.

SYNOPSIS

require Unicode::Map8;
my $no_map = Unicode::Map8->new("ISO646-NO") || die;
my $l1_map = Unicode::Map8->new("latin1") || die;

my $ustr = $no_map->to16("V}re norske tegn b|r {resn");
my $lstr = $l1_map->to8($ustr);
print $lstr;

print $no_map->tou("V}re norske tegn b|r {resn")->utf8

The Unicode::Map8 class implement efficient mapping tables between 8-bit character sets and 16 bit character sets like Unicode. The tables are efficient both in terms of space allocated and translation speed. The 16-bit strings is assumed to use network byte order.

The following methods are available:

$m = Unicode::Map8->new( [$charset] )

The object constructor creates new instances of the Unicode::Map8 class. I takes an optional argument that specify then name of a 8-bit character set to initialize mappings from. The argument can also be a the name of a mapping file. If the charset/file can not be located, then the constructor returns undef.

If you omit the argument, then an empty mapping table is constructed. You must then add mapping pairs to it using the addpair() method described below.

$m->addpair( $u8, $u16 );

Adds a new mapping pair to the mapping object. It takes two arguments. The first is the code value in the 8-bit character set and the second is the corresponding code value in the 16-bit character set. The same codes can be used multiple times (but using the same pair has no effect). The first definition for a code is the one that is used.

Consider the following example:

$m->addpair(0x20, 0x0020);
$m->addpair(0x20, 0x00A0);
$m->addpair(0xA0, 0x00A0);

It means that the character 0x20 and 0xA0 in the 8-bit charset maps to themselves in the 16-bit set, but in the 16-bit character set 0x0A0 maps to 0x20.

$m->default_to8( $u8 )

Set the code of the default character to use when mapping from 16-bit to 8-bit strings. If there is no mapping pair defined for a character then this default is substituted by to8() and recode8().

$m->default_to16( $u16 )

Set the code of the default character to use when mapping from 8-bit to 16-bit strings. If there is no mapping pair defined for a character then this default is used by to16(), tou() and recode8().

$m->nostrict;

All undefined mappings are replaced with the identity mapping. Undefined character are normally just removed (or replaced with the default if defined) when converting between character sets.

$m->to8( $ustr );

Converts a 16-bit character string to the corresponding string in the 8-bit character set.

$m->to16( $str );

Converts a 8-bit character string to the corresponding string in the 16-bit character set.

$m->tou( $str );

Same an to16() but return a Unicode::String object instead of a plain UCS2 string.

$m->recode8($m2, $str);

Map the string $str from one 8-bit character set ($m) to another one ($m2). Since we assume we know the mappings towards the common 16-bit encoding we can use this to convert between any of the 8-bit character sets.

$m->to_char16( $u8 )

Maps a single 8-bit character code to an 16-bit code. If the 8-bit character is unmapped then the constant NOCHAR is returned. The default is not used and the callback method is not invoked.

$m->to_char8( $u16 )

Maps a single 16-bit character code to an 8-bit code. If the 16-bit character is unmapped then the constant NOCHAR is returned. The default is not used and the callback method is not invoked.

The following callback methods are available. You can override these methods by creating a subclass of Unicode::Map8.

$m->unmapped_to8

When mapping to 8-bit character string and there is no mapping defined (and no default either), then this method is called as the last resort. It is called with a single integer argument which is the code of the unmapped 16-bit character. It is expected to return a string that will be incorporated in the 8-bit string. The default version of this method always returns an empty string.

Example:

package MyMapper;
@ISA=qw(Unicode::Map8);

sub unmapped_to8
{
my($self, $code) = @_;
require Unicode::CharName;
"";
}

$m->unmapped_to16

Likewise when mapping to 16-bit character string and no mapping is defined then this method is called. It should return a 16-bit string with the bytes in network byte order. The default version of this method always returns an empty string.

SWF::Builder::Character::Sound is a SWF Sound character.

SYNOPSIS

my $sound = $mc->new_sound( ring.mp3 );
$sound->play;

This module creates SWF sound characters from MP3 or raw Microsoft WAV files.

$sound = $mc->new_sound( $filename )

loads a sound file and returns a new sound character. It supports only MP3 now.

$sound->play( [ %options ] )

plays the sound.

Options:

MovieClip => $mc, Frame => $frame

MovieClip(MC) is a parent movie clip on which the sound is played. If MC is not set, the sound is played on the movie clip in which it is defined. Frame is the frame number on which the sound is played.

Multiple => 0/1

avoids/allows multiple playing. If 0, dont start the sound if already playing.

Loop => $count

sets the loop count.

In => $in_msec, Out => $out_msec

In sets the beginning point of the sound and Out sets the last in milliseconds.

Envelope => [ $msec1, $volumelevel1, $msec2, $volumelevel2, ... ]

sets the sound envelope. Volume level is set to $volumelevel1 at $msec1, and $volumelevel2 at $msec2, ... Volume level can take a number from 0 to 32768, or a reference to the array of volume levels of left and right channels.

$sound->stop( [ MovieClip => $mc, Frame => $frame ] )

stops playing the sound. It can take MovieClip and Frame options as same as the play method.

$sound->start_streaming( [ MovieClip => $mc, Frame => $frame ] )

starts the streaming sound, which synchronizes with the movie timeline. It can take MovieClip and Frame options as same as the play method.

$sound->Latency( $msec )

sets the sound latency in milliseconds.

Unicode::Unihan is the Unihan Data Base 5.0.0.

SYNOPSIS

use Unicode::Unihan;
my $db = new Unicode::Unihan;
print join("," => $db->Mandarin("x{5c0f}x{98fc}x{5f3e}"), "n";

ABSTRACT

This module provides a user-friendly interface to the Unicode Unihan Database 3.2. With this module, the Unihan database is as easy as shown in the SYNOPSIS above.

The first thing you do is make the database available. Just say

use Unicode::Unihan;
my $db = new Unicode::Unihan;

Thats all you have to say. After that, you can access the database via $db->tag($string) where tag is the tag in the Unihan Database, without k prefix.
$data = $db->tag($string) =item @data = $db->tag($string)

The first form (scalar context) returns the Unihan Database entry of the first character in $string. The second form (array context) checks the entry for each character in $string.

@data = $db->Mandarin("x{5c0f}x{98fc}x{5f3e}");
# @data is now (SHAO4 XIAO3,SI4,DAN4)

@data = $db->JapaneseKun("x{5c0f}x{98fc}x{5f3e}");
# @data is now (CHIISAI KO O,KAU YASHINAU,TAMA HAZUMU HIKU)

Unicode::MapUTF8 is a Perl module with conversions to and from arbitrary character sets and UTF8.

SYNOPSIS

use Unicode::MapUTF8 qw(to_utf8 from_utf8 utf8_supported_charset);

# Convert a string in ISO-8859-1 to UTF8
my $output = to_utf8({ -string => An example, -charset => ISO-8859-1 });

# Convert a string in UTF8 encoding to encoding ISO-8859-1
my $other = from_utf8({ -string => Other text, -charset => ISO-8859-1 });

# List available character set encodings
my @character_sets = utf8_supported_charset;

# Add a character set alias
utf8_charset_alias({ ms-japanese => sjis });

# Convert between two arbitrary (but largely compatible) charset encodings
# (SJIS to EUC-JP)
my $utf8_string = to_utf8({ -string =>$sjis_string, -charset => sjis});
my $euc_jp_string = from_utf8({ -string => $utf8_string, -charset => euc-jp })

# Verify that a specific character set is supported
if (utf8_supported_charset(ISO-8859-1) {
# Yes
}

Provides an adapter layer between core routines for converting to and from UTF8 and other encodings. In essence, a way to give multiple existing Unicode modules a single common interface so you dont have to know the underlaying implementations to do simple UTF8 to-from other character set encoding conversions. As such, it wraps the Unicode::String, Unicode::Map8, Unicode::Map and Jcode modules in a standardized and simple API.

This also provides general character set conversion operation based on UTF8 - it is possible to convert between any two compatible and supported character sets via a simple two step chaining of conversions.

As with most things Perlish - if you give it a few big chunks of text to chew on instead of lots of small ones it will handle many more characters per second.

By design, it can be easily extended to encompass any new charset encoding conversion modules that arrive on the scene.
This module is intended to provide good Unicode support to versions of Perl prior to 5.8. If you are using Perl 5.8.0 or later, you probably want to be using the Encode module instead. This module does work with Perl 5.8, but Encode is the preferred method in that environment.

Encode::HanExtra Perl module contains extra sets of Chinese encodings.



SYNOPSIS

use Encode;

# Traditional Chinese
$euc_tw = encode("euc-tw", $utf8); # loads Encode::HanExtra implicitly
$utf8 = decode("euc-tw", $euc_tw); # ditto

# Simplified Chinese
$gb18030 = encode("gb18030", $utf8); # loads Encode::HanExtra implicitly
$utf8 = decode("gb18030", $gb18030); # ditto

Perl 5.7.3 and later ships with an adequate set of Chinese encodings, including the commonly used CP950, CP936 (also known as GBK), Big5 (alias for Big5-Eten), Big5-HKSCS, EUC-CN, HZ, and ISO-IR-165.

However, the numbers of Chinese encodings are staggering, and a complete coverage will easily increase the size of perl distribution by several megabytes; hence, this CPAN module tries to provide the rest of them.

If you are using perl 5.8 or later, Encode::CN and Encode::TW will automatically load the extra encodings for you, so theres no need to explicitly write use Encode::HanExtra if you are using one of them already.

ENCODINGS

This version includes the following encoding tables:

Canonical Alias Description
-----------------------------------------------------------------------------
big5-1984 /b(tca-)?big5-?(19)?84$/i TCAs original Big5-1984
big5ext /b(cmex-)?big5-?e(xt)?$/i CMEXs Big5e Extension
big5plus /b(cmex-)?big5-?p(lus)?$/i CMEXs Big5+ Extension
/b(cmex-)?big5+$/i
cccii /b(ccag-)?cccii$/i Chinese Character Code for
Information Interchange
cns11643-1 /bCNS[-_ ]?11643[-_]1$/i Taiwans CNS map, plane 1
cns11643-2 /bCNS[-_ ]?11643[-_]2$/i Taiwans CNS map, plane 2
cns11643-3 /bCNS[-_ ]?11643[-_]3$/i Taiwans CNS map, plane 3
cns11643-4 /bCNS[-_ ]?11643[-_]4$/i Taiwans CNS map, plane 4
cns11643-5 /bCNS[-_ ]?11643[-_]5$/i Taiwans CNS map, plane 5
cns11643-6 /bCNS[-_ ]?11643[-_]6$/i Taiwans CNS map, plane 6
cns11643-7 /bCNS[-_ ]?11643[-_]7$/i Taiwans CNS map, plane 7
cns11643-f /bCNS[-_ ]?11643[-_]f$/i Taiwans CNS map, plane F
euc-tw /beuc.*tw$/i EUC (Extended Unix Character)
/btw.*euc$/i
gb18030 /bGB[-_ ]?18030$/i GBK with Traditional Characters
unisys /bunisys$/i Unisys Traditional Chinese
unisys-sosi1 Unisys SOSI1 transport encoding
unisys-sosi2 Unisys SOSI2 transport encoding

Detailed descriptions are as follows:

BIG5-1984

This is the original Big5 encoding made by TCA Taiwan.

BIG5PLUS

This encoding, while not heavily used, is an attempt to bring all Taiwans conflicting internal-use encodings together, and fit it as an extension to the widely-deployed Big5 range, by CMEX Taiwan.

BIG5EXT

The CMEXs second (and less ambitious) try at unifying the most commonly used characters not covered by Big5, while not polluting out of the 94x94 arragement like BIG5PLUS did.

CCCII

The earliest (and most sophisticated) Traditional Chinese encoding, with a three-byte raw character map, made in 1980 by the Chinese Character Analysis Group (CCAG), used mostly in library systems.

EUC-TW

The EUC transport version of CNS11643 (planes 1-7), the comprehensive character set used by the Taiwan government.

CNS11643-*

The raw character map extracted from the Unihan database, including the plane F which wasnt included in EUC-TW.

GB18030

An extension to GBK, this encoding lists most Han characters (both simplified and traditional), as well as some other encodings used by other peoples in China.

UNISYS

Unisys Systems internal Chinese mapping.

ustring is a C++ library to implement the complete Unicode 3.0 standard. It gives an easy access to the properties of each of the 10600 characters defined by the Unicode 3.0 standard, including the uppercase and lowercase mapping, category and subcategory, character name, and bidirectional category.

ustring C++ Unicode library provides a wide string class which fits with STL, implementing a full Unicode normalizer, uppercase and lowercase transformation, and diacritical mark removal. A Unicode level-1 regular expression class is also provided.