Free text software for linux

GD::Text is a Perl module with text utilities for use with GD.

SYNOPSIS

use GD;
use GD::Text;

my $gd_text = GD::Text->new() or die GD::Text::error();
$gd_text->set_font(funny.ttf, 12) or die $gd_text->error;
$gd_text->set_font(gdTinyFont);
$gd_text->set_font(GD::Font::Tiny);
...
$gd_text->set_text($string);
my ($w, $h) = $gd_text->get(width, height);

if ($gd_text->is_ttf)
{
...
}
Or alternatively
my $gd_text = GD::Text->new(
text => Some text,
font => funny.ttf,
ptsize => 14,
);

This module provides a font-independent way of dealing with text in GD, for use with the GD::Text::* modules and GD::Graph.

Text::CPP is a full C Preprocessor in XS.

SYNOPSIS

use Text::CPP;
my $reader = new Text::CPP(
Language => CLK_GNUC99,
Options => {
...
},
Builtins => {
foo => this,
bar => that,
},
);
$reader->read("file.c");
while (my $token = $reader->token) {
print "Token: $tokenn";
}

$reader->data->{MyKey} = $MyData;

A fast C preprocessor in XS. This does not require an external C preprocessor, and will not fork() or exec() any external process.

USAGE

The following methods have been implemented, allowing the use of this module as a pure C preprocessor, or as a lexer for a C, C++ or Assembler-like language.

new Text::CPP(...)

Takes a hash or hashref with the following possible keys:

Language

Defines the source language to preprocess and/or tokenise. It may be any of:

CLK_GNUC89 - GNU C89
CLK_GNUC99 - GNU C99
CLK_STDC89 - Standard C89
CLK_STDC94 - Standard C94
CLK_STDC99 - Standard C99
CLK_GNUCXX - GNU C++
CLK_CXX98 - Standard C++ 98
CLK_ASM - Assembler

Options

A hashref of options for the preprocessor. Valid entries are given with alternative forms (from GNU cpp) in brackets.

Define (-D): array of strings or hash

Strings should be of the form NAME=VALUE.

Undef (-U): array of strings

DiscardComments (-C): boolean

DiscardCommentsInMacroExp (-CC): boolean

PrintIncludeNames (-H): boolean

NoLineCommands (-P): boolean

WarnComments (-Wcomment -Wcomments): boolean

WarnDeprecated (-Wdeprecated): boolean

WarningsAreErrors (-Werror): boolean

WarnImport (-Wimport): boolean

WarnMultichar (-Wmultichar): boolean

WarnSystemHeaders (-Wsystem-headers): boolean

Ignore errors in system header files.

WarnTraditional (-Wtraditional): boolean

WarnTrigraphs (-Wtrigraphs): boolean

WarnUnusedMacros (-Wunused-macros): boolean

Pedantic (-pedantic): boolean

PedanticErrors (-pedantic-errors): boolean

Implies, and overrides, Pedantic.

Remap (-remap): boolean

Deal with some brokennesses of MSDOS. Untested.

Trigraphs (-trigraphs): boolean

Traditional (-traditional): boolean

NoWarnings (-w): boolean

IncludePrefix (-iprefix): string

SystemRoot (-isysroot): string

Include (-include): array of strings

Include the specified files before reading the main file to be processed.

IncludeMacros (-imacros): array of strings

Include the specified files before reading the main file to be processed. Output from preprocessing these files is discarded. Files specified by IncludeMacros are processed before files specified by Include.

IncludePath (-I): array of strings

This include path is searched first.

SystemIncludePath (-isystem): array of strings

Specify the standard system include path, searched second.

AfterIncludePath (-idirafter): array of strings

This include path is searched after the system include path.

Builtins

A hashref of predefined macros. The values must be strings or integers. Macros in this hash will be defined before preprocessing starts. These correspond to true "builtin" macros. You should probably prefer to use the Define option.

$text = $reader->token

($text, $type, $flags) = $reader->token

Return the next available preprocessed token. Some tokens are not stringifiable. These include tokens of type CPP_MACRO_ARG, CPP_PADDING and CPP_EOF. Text::CPP returns a dummy string in the text field for these tokens. Tokens of type CPP_EOF should never actually be returned.

@tokens = $reader->tokens

Preprocess and return a list of tokens. This is approximately equivalent to:

push(@tokens, $_) while ($_ = $reader->token);

$reader->type($type)

Return a human readable name for a token type, as returned by $reader->token.

$reader->data

Returns a hashref in which user data may be stored by subclasses. This hashref is created with a new Text::CPP object, and is ignored for all functional purposes. The user may do with it as he wishes.

$reader->errors

In scalar context, returns the fatal error count. In list context, returns a list of warnings and errors encountered by the preprocessor. Thus scalar(@errors) >= $errors, since @errors will also contain the warnings.

Text::Quote contains quotes strings as required for perl to eval them back correctly.

SYNOPSIS

use Text::Quote;

my @quotes=map{$quoter->quote($_,indent=>6,col_width=>60)}(
"The time has come"
the walrus said,
"to speak of many things..."
," 123456abtn13fr16172021222324252627303132e34353637",
("6abtn13fr32e34" x 5),2/3,10,00);
for my $i (1..@quotes) {
print "$var$i=".$quotes[$i-1].";n";
}
Would produce:
$var1=qq"The time has come"ntthetwalrus said,nt"to speak of man.
qqy things...";
$var2=" 123456abtn13fr16172021222324252627".
"303132e34353637";
$var3=("6abtn13fr32e34" x 5);
$var4=0.666666666666667;
$var5=10;
$var6=00;

Text::Ngrams is a flexible Ngram analysis (for characters, words, and more).
SYNOPSIS
For default character n-gram analysis of string:
use Text::Ngrams;
my $ng3 = Text::Ngrams->new;
$ng3->process_text(abcdefg1235678hijklmnop);
print $ng3->to_string;
my @ngramsarray = $ng3->get_ngrams;
One can also feed tokens manually:
use Text::Ngrams;
my $ng3 = Text::Ngrams->new;
$ng3->feed_tokens(a);
$ng3->feed_tokens(b);
$ng3->feed_tokens(c);
$ng3->feed_tokens(d);
$ng3->feed_tokens(e);
$ng3->feed_tokens(f);
$ng3->feed_tokens(g);
$ng3->feed_tokens(h);
We can choose n-grams of various sizes, e.g.:
my $ng = Text::Ngrams->new( windowsize => 6 );
or different types of n-grams, e.g.:
my $ng = Text::Ngrams->new( type => byte );
my $ng = Text::Ngrams->new( type => word );
my $ng = Text::Ngrams->new( type => utf8 );
To process a list of files:
$ng->process_files(somefile.txt, otherfile.txt);
This module implement text n-gram analysis, supporting several types of analysis, including character and word n-grams.
The module Text::Ngrams is very flexible. For example, it allows a user to manually feed a sequence of any tokens. It handles several types of tokens (character, word), and also allows a lot of flexibility in automatic recognition and feed of tokens and the way they are combined in an n-gram. It counts all n-gram frequencies up to the maximal specified length. The output format is meant to be pretty much human-readable, while also loadable by the module.
The module can be used from the command line through the script ngrams.pl provided with the package.
Version restrictions:
- If a user customizes a type, it is possible that a resulting n-gram will be ambiguous. In this way, to different n-grams may be counted as one. With predefined types of n-grams, this should not happen. For example, if a user chooses that a token can contain a space, and uses space as an n-gram separator, then a trigram like this "x x x x" is ambiguous.
- Method process_file does not handle multi-line tokens by default. This can be fixed, but it does not seem to be worth the code complication. There are various ways around this if one really needs such tokens: One way is to preprocess them. Another way is to read as much text as necessary at a time then to use process_text, which does handle multi-line tokens.

Text::Indent is a simple indentation of text shared among modules.

SYNOPSIS

In your main program:

use Text::Indent;
my $indent = Text::Indent->new;
$indent->spaces(2);

In a module to produce indented output:

use Text::Indent;
my $indent = Text::Indent->instance;
$indent->increase;
print $indent->indent("this will be indented two spaces");
$indent->increase(2);
print $indent->indent("this will be indented six spaces");
$indent->decrease(3);

Text::Indent is designed for use in programs which need to produce output with multiple levels of indent when the source of the output comes from different modules that know nothing about each other.

For example take module A, whose output includes the indented output of module B. Module B can also produce output directly, so it falls to module B to know whether it should indent its output or not depending on its calling context.

Text::Indent allows programs and modules to cooperate to choose an appropriate indent level that is shared within the program context. In the above example, module A would increase the indent level prior to calling the output routines of module B. Module B would simply use the Text::Indent instance confident that if it were being called directly no indent would be applied but if module A was calling it then its output would be indented one level.

Text::Macro Perl module is a template facility whos focus is on generating code such as c, java or sql. While generating perl code is also possible, there is a potential conflict between the control-symbol and the perl comment symbol.
Perl is excelent at manipulating text, and it begs the question why one would need such a tool.
The answer is that good code design should be such that applications should not have to be modified so as to make configuration changes. Thus external configuration files/data is used. However, if these files are read in as perl-code, then simple errors could crash the whole application (or provide subtle security risks). Further, it is often desired to invert the control flow and text-data (namely, make the embedded strings primary, and control-flow secondary). This is the ASP model, and for 90% HTML, 10% code, this works great.
This module supports many control facilities which directly translate into perl-control facilities (e.g. inverting the ASP-style code back into perl-style behind the scenes). The inversion process is cached in a simple user object.
The module was initially inspired by Text::FastTemplate by Robert Lehr, whos module didnt completely fullfill my needs.
Main features:
- fast, simple, robust
- code-generating-centric feature-set
- substitutions stand-out from template
- macro-code embedded in text
- OOP
- external and internal includes (for clearifying complex control-flow)
- scoped variable-substitutions
- line-based processing (like cpp)
- usable error messages

Text::Bloom can evaluate Bloom signature of a set of terms.

SYNOPSIS

my $b = Text::Bloom->new();
$b->Compute( qw( foo bar baz ) );
my $sig = $b->WriteToString();
$b->WriteToFile( afile.sig );
my $b2 = Text::Bloom::NewFromFile( afile.sig );
my $b3 = Text::Bloom->new();
$b3->Compute( qw( foo bar barbaz ) );
my $sim = $b->Similarity( $b2 );
my $b4 = Text::Bloom::NewFromString( $sig );

Text::Bloom applies the Bloom filtering technique to the statistical analysis of documents.

The terms in the document are quantized using a base-36 radix representation; each term thus corresponds to an integer in the range 0..p-1, where p is a prime, currently set to the greatest prime less than 2^32.

Each quantized value is mapped to d integers in the range 0..size-1, where size is an integer less than p, currently 2^17, using a family of hash functions, computed by the HashV function.

Each hashed value is used as the index in a large bit vector. Bits corresponding to terms present in the document are set to 1; all other bits are set to 0.

Of course, collisions may cause the same bit to be set twice, by different terms. It follows that, if the document contains n distinct terms, in the resulting bit vector at most n * d bits are set to 1.

The resulting bit string is a very compact representation of the presence/absence of terms in the document, and is therefore characterised as a signature. Moreover, it does not depend on a pre-set dictionary of terms.

The signature may be used for:

testing whether a given set of terms is present in the document,
computing which fraction of terms are common to two documents.

The bit representation may be written to and read from a file. Text::Bloom prepends a header to the bit stream proper; moreover, whenever the package Compress::Zlib is available, the bit vector is compressed, so that disk space requirements are drastically reduced, especially for small documents.

The hash function is obviously a crucial component of the filter; the reference implementation uses a radix representation of strings. Each term must therefore match the regular expression /[0-9a-z]+/.

There are quite a few viable alternatives, which can be pursued by subclassing and redefining the method QuantizeV.