Free lingua ru charset software for linux

MARC::Charset is a Perl module to convert MARC-8 encoded strings to UTF-8.

SYNOPSIS

# import the marc8_to_utf8 function
use MARC::Charset marc8_to_utf8;

# prepare STDOUT for utf8
binmode(STDOUT, utf8);

# print out some marc8 as utf8
print marc8_to_utf8($marc8_string);

MARC::Charset allows you to turn MARC-8 encoded strings into UTF-8 strings. MARC-8 is a single byte character encoding that predates unicode, and allows you to put non-Roman scripts in MARC bibliographic records.

Math::String::Charset is a simple charset for Math::String objects.

SYNOPSIS

use Math::String::Charset;

$a = new Math::String::Charset; # default a-z
$b = new Math::String::Charset [a..z]; # same
$c = new Math::String::Charset
{ start => [a..z], sep => }; # with between chars

print $b->length(); # a-z => 26

# construct a charset from bigram table, and an initial set (containing
# valid start-characters)
# Note: After an a, either an b, c or a can follow, in this order
# After an d only an a can follow
$bi = new Math::String::Charset ( {
start => a..d,
bi => {
a => [ b, c, a ],
b => [ c, b ],
c => [ a, c ],
d => [ a, ],
q => [ ], # q will be automatically in end
}
end => [ a, b, ],
} );
print $bi->length(); # a,b => 2 (cross of end and start)
print scalar $bi->class(2); # count of combinations with 2 letters
# will be 3+2+2+1 => 8

$d = new Math::String::Charset ( { start => [a..z],
minlen => 2, maxlen => 4, } );

print $d->first(0),"n"; # undef, too short
print $d->first(1),"n"; # undef, to short
print $d->first(2),"n"; # aa

$d = new Math::String::Charset ( { start => [a..z] } );

print $d->first(0),"n"; #
print $d->first(1),"n"; # a
print $d->last(1),"n"; # z
print $d->first(2),"n"; # aa

This module lets you create an charset object, which is used to contruct Math::String objects. This object knows how to handle simple charsets as well as complex onex consisting of bi-grams (later tri and more).

In case of more complex charsets, a reference to a Math::String::Charset::Nested or Math::String::Charset::grouped will be returned.

The default charset is the set containing "abcdefghijklmnopqrstuvwxyz" (thus producing always lower case output).

MIME::Charset module contains charset information for MIME.

SYNOPSIS

Getting charset informations:

use MIME::Charset qw(:info);

$benc = body_encoding("iso-8859-2"); # "Q"
$cset = canonical_charset("ANSI X3.4-1968"); # "US-ASCII"
$henc = header_encoding("utf-8"); # "S"
$cset = output_charset("shift_jis"); # "ISO-2022-JP"

Translating text data:

use MIME::Charset qw(:trans);

($text, $charset, $encoding) =
header_encode(
"xc9xc2xc5xaaxc0xdexc3xefxc5xaa".
"xc7xd1xcaxaaxbdxd0xcexcfxb4xef",
"euc-jp");
# ...returns (< converted >, "ISO-2022-JP", "B");

($text, $charset, $encoding) =
body_encode(
"Collectioneur pathxe9tiquement ".
"xe9clectique de dxe9chets",
"latin1");
# ...returns (< original >, "ISO-8859-1", "QUOTED-PRINTABLE");

$len = encoded_header_len(
"Perlxe8xa8x80xe8xaax9e", "b", "utf-8"); # 28

Manipulating module defaults:

use MIME::Charset;

MIME::Charset::alias("csEUCKR", "euc-kr");
MIME::Charset::default("iso-8859-1");
MIME::Charset::fallback("us-ascii");

Lingua::Phonology::Segment is a module to represent a segment as a bundle of feature values.

SYNOPSIS

use Lingua::Phonology;
$phono = new Lingua::Phonology;

# Define a feature set
$features = $phono->features;
$features->loadfile;

# Make a segment
$segment = $phono->segment;

# Set some values
$segment->labial(1);
$segment->continuant(0);
$segment->voice(1);
# Segment is now voiced labial stop, i.e. [b]

# Reset the segment
$segment->clear;

A Lingua::Phonology::Segment object provides a programmatic representation of a linguistic segment. Such a segment is associated with a Lingua::Phonology::Features object that lists the available features and the relationships between them. The segment itself is a list of the values for those features.

This module provides methods for returning and setting these feature values. A segment may also be associated with a Lingua::Phonology::Symbols object, which allows the segment to return the symbol that it best matches.

OVERLOADING

As of Lingua::Phonology v0.32 (Lingua::Phonology::Segment v0.4), string conversion of segments is overloaded. When you use a Lingua::Phonology::Segment in string context, the spell() method is automatically called, and the representation of the segment from the current symbolset is returned. String comparison operators (cmp eq ne lt le gt ge) are also overloaded. Therefore, the following work correctly, assuming that you have a Lingua::Phonology object correctly set up in $phono.

my ($b, $k) = $phono->symbols->segment(b, k);

print "Segments: $b, $kn"; # Prints "Segments: b, k";
print "$b is greater than $kn" if $b gt $k; # Wont print
print "$b is less than $kn" if $b lt $k; # Prints b is less than k;
print "$b is equal to $kn" if $b eq $k; # Wont print

my $b2 = $b->duplicate;
print "$b is equal to $b2n" if $b eq $b2 # Prints b is equal to b;
Note that stringification is not overloaded if the symbolset is not properly set. However, it turns on as soon as a symbolset is available:
my $b = Lingua::Phonology::Segment->new($features);
$b->voice(1);
$b->labial(1);

print "$bn"; # Prints Lingua::Phonology::Segment=HASH(0x88af598)
# or something similar, because there is no symbolset
# defined for spelling the segment.

$b->symbolset($symbols);
print "$bn"; # Prints b
Number conversion is not overloaded.

VarCon is a data set provided as part of the Word List project.

It contains a number of lists and tables of words that can be used to help automatically convert the English language between the differing spellings of its local dialects (American, British, Canadian).

In an age where Spanish application translations are common, but finding a British, Canadian or Australian translation (as used by FAR more people than American English) can be almost impossible, and when "English" on the internet is increasingly seeming to be "American", it is hoped that by providing access to the raw data in a variety of different ways (optimised for different application types with different load profiles) it will encourage developers to integrate support for automated dialect translation into internationalisation toolkits, and thus into many applications.

METHODS

For this initial release, only methods to locate the files are provided.

Additional access methods will be provided later, or on request.

abbc_file

The abbc_file method returns the location of the abbc.tab file from the VarCon data set.

also_file

The also_file method returns the location of the variant-also.tab file from the VarCon data set.

infl_file

The infl_file method returns the location of the variant-infl.tab file from the VarCon data set.

wroot_file

The infl_file method returns the location of the variant-wroot.tab file from the VarCon data set.

voc_file

The voc_file method returns the location of the voc.tab file from the VarCon data set.

Lingua::Phonology::Rules is a Perl module for defining and applying phonological rules.

SYNOPSIS

use Lingua::Phonology;
$phono = new Lingua::Phonology;

$rules = $phono->rules;

# Adding and manipulating rules is discussed in the "WRITING RULES"
# section

This module allows for the creation of linguistic rules, and the application of those rules to "words" of Segment objects. You, the user, add rules to a Rules object, defining various parameters and code references that actually perform the action of the rule. Lingua::Phonology::Rules will take care of the guts of applying and creating rules.

The rules you create may have the following parameters. This is just a brief description of the parameters--a more detailed discussion of their effect is in the "WRITING RULES" section.

domain

Defines the domain within which the rule applies. This should be the name of a feature in the featureset of the segments which the rule is applied to.

tier

Defines the tier on which the rule applies. Must be the name of a feature in the feature set for the segments of the word you pass in.

direction

Defines the direction that the rule applies in. Must be either leftward or rightward. If no direction is given, defaults to rightward.

filter

Defines a filter for the segments that the rule applies on. Must a code reference that returns a truth value.

linguistic

Defines a linguistic-style rule to be parsed. When you provide a linguistic-style rule, it is parsed into code references that take the place of the where and do properties listed below. The format of linguistic rules is described in "LINGUISTIC-STYLE RULES" in Lingua::Phonology::FileFormatPOD.

where - defines the condition or conditions where the rule applies. Must be a coderef that returns a truth value. If no value is given, defaults to always true.

do - defines the action to take when the where condition is met. Must be a code reference. If no value is given, does nothing.

result - EXPERIMENTAL. Defines a condition that must be true after the do code has applied. Must be a code reference that returns a truth value. NOTE: This parameter depends on the module Whatif (available from CPAN), and will behave differently if this module is not present. See "Using result".

Lingua::Phonology::Rules is flexible and powerful enough to handle any sequential type of rule system. It cannot handle Optimality Theory-style processes, because those require a fundamentally different kind of algorithm.

Lingua::EN::Inflect is a Perl module that can convert singular to plural. Select "a" or "an".

SYNOPSIS

use Lingua::EN::Inflect qw ( PL PL_N PL_V PL_ADJ NO NUM
PL_eq PL_N_eq PL_V_eq PL_ADJ_eq
A AN
PART_PRES
ORD NUMWORDS
inflect classical
def_noun def_verb def_adj def_a def_an );


# UNCONDITIONALLY FORM THE PLURAL

print "The plural of ", $word, " is ", PL($word), "n";


# CONDITIONALLY FORM THE PLURAL

print "I saw $cat_count ", PL("cat",$cat_count), "n";


# FORM PLURALS FOR SPECIFIC PARTS OF SPEECH

print PL_N("I",$N1), PL_V("saw",$N1),
PL_ADJ("my",$N2), PL_N("saw",$N2), "n";


# DEAL WITH "0/1/N" -> "no/1/N" TRANSLATION:

print "There ", PL_V("was",$errors), NO(" error",$errors), "n";


# USE DEFAULT COUNTS:

print NUM($N1,""), PL("I"), PL_V(" saw"), NUM($N2), PL_N(" saw");
print "There ", NUM($errors,), PL_V("was"), NO(" error"), "n";


# COMPARE TWO WORDS "NUMBER-INSENSITIVELY":

print "samen" if PL_eq($word1, $word2);
print "same nounn" if PL_eq_N($word1, $word2);
print "same verbn" if PL_eq_V($word1, $word2);
print "same adj.n" if PL_eq_ADJ($word1, $word2);


# ADD CORRECT "a" OR "an" FOR A GIVEN WORD:

print "Did you want ", A($thing), " or ", AN($idea), "n";


# CONVERT NUMERALS INTO ORDINALS (i.e. 1->1st, 2->2nd, 3->3rd, etc.)

print "It was", ORD($position), " from the leftn";

# CONVERT NUMERALS TO WORDS (i.e. 1->"one", 101->"one hundred and one", etc.)
# IN A SCALAR CONTEXT: GET BACK A SINGLE STRING...

$words = NUMWORDS(1234); # "one thousand, two hundred and thirty-four"
$words = NUMWORDS(ORD(1234)); # "one thousand, two hundred and thirty-fourth"


# IN A LIST CONTEXT: GET BACK A LIST OF STRINGSi, ONE FOR EACH "CHUNK"...

@words = NUMWORDS(1234); # ("one thousand","two hundred and thirty-four")


# OPTIONAL PARAMETERS CHANGE TRANSLATION:

$words = NUMWORDS(12345, group=>1);
# "one, two, three, four, five"

$words = NUMWORDS(12345, group=>2);
# "twelve, thirty-four, five"

$words = NUMWORDS(12345, group=>3);
# "one twenty-three, forty-five"

$words = NUMWORDS(1234, and=>);
# "one thousand, two hundred thirty-four"

$words = NUMWORDS(1234, and=>, plus);
# "one thousand, two hundred, plus thirty-four"

$words = NUMWORDS(555_1202, group=>1, zero=>oh);
# "five, five, five, one, two, oh, two"

$words = NUMWORDS(555_1202, group=>1, one=>unity);
# "five, five, five, unity, two, oh, two"

$words = NUMWORDS(123.456, group=>1, decimal=>mark);
# "one two three mark four five six"


# REQUIRE "CLASSICAL" PLURALS (EG: "focus"->"foci", "cherub"->"cherubim")

classical; # USE ALL CLASSICAL PLURALS

classical 1; # USE ALL CLASSICAL PLURALS
classical 0; # USE ALL MODERN PLURALS (DEFAULT)

classical zero; # "no error" INSTEAD OF "no errors"
classical zero=>1; # "no error" INSTEAD OF "no errors"
classical zero=>0; # "no errors" INSTEAD OF "no error"

classical herd; # "2 buffalo" INSTEAD OF "2 buffalos"
classical herd=>1; # "2 buffalo" INSTEAD OF "2 buffalos"
classical herd=>0; # "2 buffalos" INSTEAD OF "2 buffalo"

classical persons; # "2 chairpersons" INSTEAD OF "2 chairpeople"
classical persons=>1; # "2 chairpersons" INSTEAD OF "2 chairpeople"
classical persons=>0; # "2 chairpeople" INSTEAD OF "2 chairpersons"

classical ancient; # "2 formulae" INSTEAD OF "2 formulas"
classical ancient=>1; # "2 formulae" INSTEAD OF "2 formulas"
classical ancient=>0; # "2 formulas" INSTEAD OF "2 formulae"



# INTERPOLATE "PL()", "PL_N()", "PL_V()", "PL_ADJ()", A()", "AN()"
# "NUM()" AND "ORD()" WITHIN STRINGS:

print inflect("The plural of $word is PL($word)n");
print inflect("I saw $cat_count PL("cat",$cat_count)n");
print inflect("PL(I,$N1) PL_V(saw,$N1) PL(a,$N2) PL_N(saw,$N2)");
print inflect("NUM($N1,)PL(I) PL_V(saw) NUM($N2,)PL(a) PL_N(saw)");
print inflect("I saw NUM($cat_count) PL("cat")nNUM()");
print inflect("There PL_V(was,$errors) NO(error,$errors)n");
print inflect("There NUM($errors,) PL_V(was) NO(error)n";
print inflect("Did you want A($thing) or AN($idea)n");
print inflect("It was ORD($position) from the leftn");


# ADD USER-DEFINED INFLECTIONS (OVERRIDING INBUILT RULES):

def_noun "VAX" => "VAXen"; # SINGULAR => PLURAL

def_verb "will" => "shall", # 1ST PERSON SINGULAR => PLURAL
"will" => "will", # 2ND PERSON SINGULAR => PLURAL
"will" => "will", # 3RD PERSON SINGULAR => PLURAL

def_adj "hir" => "their", # SINGULAR => PLURAL

def_a "h" # "AY HALWAYS SEZ HAITCH!"

def_an "horrendous.*" # "AN HORRENDOUS AFFECTATION"

The exportable subroutines of Lingua::EN::Inflect provide plural inflections, "a"/"an" selection for English words, and manipulation of numbers as words

Plural forms of all nouns, most verbs, and some adjectives are provided. Where appropriate, "classical" variants (for example: "brother" -> "brethren", "dogma" -> "dogmata", etc.) are also provided.

Pronunciation-based "a"/"an" selection is provided for all English words, and most initialisms.
It is also possible to inflect numerals (1,2,3) to ordinals (1st, 2nd, 3rd) and to english words ("one", "two", "three).

In generating these inflections, Lingua::EN::Inflect follows the Oxford English Dictionary and the guidelines in Fowlers Modern English Usage, preferring the former where the two disagree.
The module is built around standard British spelling, but is designed to cope with common American variants as well. Slang, jargon, and other English dialects are not explicitly catered for.

Where two or more inflected forms exist for a single word (typically a "classical" form and a "modern" form), Lingua::EN::Inflect prefers the more common form (typically the "modern" one), unless "classical" processing has been specified (see "MODERN VS CLASSICAL INFLECTIONS").