Free text processing software for linux

Docutils is an open-source text processing system for processing plaintext documentation into useful formats, such as HTML or LaTeX.

Docutils project includes reStructuredText, the easy to read, easy to use, what-you-see-is-what-you-get plaintext markup language.

MKDoc::Text::Structured::Inline is a Perl module to convert text to HTML without handling block-level tags.

SYNOPSIS

my $text = some_structured_text();
my $this = MKDoc::Text::Structured::Inline::process ($text);
my $that = MKDoc::Text::Structured::Inline::process_entities_only ($text);

Text::MetaText is a Perl extension implementing meta-language for processing "template" text files.

SYNOPSIS

use Text::MetaText;

my $mt = Text::MetaText->new();

# process file content or text string
print $mt->process_file($filename, %vardefs);
print $mt->process_text($textstring, %vardefs);

# pre-declare a BLOCK for subsequent INCLUDE
$mt->declare($textstring, $blockname);
$mt->declare(@content, $blockname);

SUMMARY OF METATEXT DIRECTIVES

%% DEFINE
variable1 = value # define variable(s)
variable2 = "quoted value"
%%

%% SUBST variable %% # insert variable value
%% variable %% # short form of above

%% BLOCK blockname %% # define a block blockname
block text...
%% ENDBLOCK %%

%% INCLUDE blockname %% # include blockname block text
%% INCLUDE filename %% # include external file filename

%% INCLUDE file_or_block # a more complete example...
variable = value # additional variable definition(s)
if = condition # conditional inclusion
unless = condition # conditional exclusion
format = format_string # printf-like format string with %s
filter = fltname(params) # post-process filter
%%

%% TIME # current system time, as per time(2)
format = format_string # display format, as per strftime(3C)
%%

Emdros is an Open-Source text database engine for storage and retrieval of analyzed or annotated text.
Emdros has a powerful query-language for asking relevant questions of the data.
Emdros has wide applicability in fields that deal with analyzed or annotated text. Application domains include linguistics, publishing, text processing, and any other fields that deal with annotated text.
Main features:
- Linguistic analyses are the primary target domain. This includes all levels of analysis, such as morphology, syntax, and discourse analysis, and even phonology to some extent.
- Publishing is also a field where Emdros can be useful. Emdros supports breaking a text down into pages, chapters, paragraphs, etc.
- Text processing may benefit from Emdros if the problem involves annotating the text.
Emdros provides a conceptual model of text which can be quite liberating to use once it has been grasped.
Meta-data may also be stored, so long as there is some textual element with which it can be associated.
Emdros is good both for corpus linguistics (large amounts of text) and for field-linguistics (smaller amounts of data).
Fixed corpora, such as Biblical texts, are good candidates for making Emdros useful. Emdros is currently being used for large databases of the Hebrew Bible.
Dictionaries are also a target possibility. Emdros supports structuring of text documents down to minute details, while not losing the big picture.
Emdros embodies a particular model of text called the EMdF model. The primary advantage over XMLs data model is that object types (such as pages and chapters) need not be hierarchically structured or embedded, but may overlap. In addition, objects (such as a clause or a phrase) need not be contiguous, but may have gaps.
Emdros can output its results in XML. The XML carries its own standalone DTD and validates with a validating parser.
Emdros architecture
Emdros fits into a software architecture as follows:
+---------------+
| Client | User-written
+---------------+
|
+---------------+
| MQL | Emdros
+---------------+
|
+---------------+
| EMdF | Emdros
+---------------+
|
+---------------+
| DB | PostgreSQL or MySQL
+---------------+
At the top, there is a client which you, the user, must write. This client will take advantage of Emdross services to provide for the needs of your particlar database domain.
Then come the two Emdros-layers: The MQL layer and the EMdF layer. The MQL layer provides an interface to the MQL language. The MQL layer automatically takes advantage of the EMdF layer, which translates the MQL queries into SQL calls to the underlying database.
The underlying database takes care of storing the data, and retrieving it as directed by the EMdF layer.
The data domain which Emdros handles is that of text. Emdros provides a certain abstraction of text that makes it ideally suited to storing and retrieving annotated text, such as linguistic analyses of a text.
These analyses can be, e.g., syntactic analyses, morphological analyses, or discourse analyses, or all of these. Phonological analyses are also supported to some extent.
Emdros is particularly useful in domains where research questions need to be asked of databases of annotated text. This would include dictionary-making, Biblical language-research (Greek or Hebrew), other linguistic research, and research on annotated text in general.
Emdros has a particular model of text called the EMdF model. Users have attested, and our experience shows, that the EMdF model can be quite liberating when dealing with text as a programmer or program designer. Thus any application that deals with annotated text will likely benefit from the Emdros and the EMdF model.
Enhancements:
- The topographic part of the language was expanded.
- A TIGER XML importer was added.
- Beta quality C# bindings were added.
- Bugfixes were made, and the regression test suite was enhanced.

Text::MacroScript is a macro pre-processor with embedded perl capability.

SYNOPSIS

use Text::MacroScript ;

# new() for macro processing

my $Macro = Text::MacroScript->new ;
while( ) {
print $Macro->expand( $_ ) if $_ ;
}

# Canonical use (the filename improves error messages):
my $Macro = Text::MacroScript->new ;
while( ) {
print $Macro->expand( $_, $ARGV ) if $_ ;
}

# new() for embedded macro processing

my $Macro = Text::MacroScript->new( -embedded => 1 ) ;
# Delimiters default to
# or
my $Macro = Text::MacroScript->new( -opendelim => [[, -closedelim => ]] ) ;
while( ) {
print $Macro->expand_delimited( $_, $ARGV ) if $_ ;
}

# Create a macro object and create initial macros/scripts from the file(s)
# given:
my $Macro = Text::MacroScript->new(
-file => [ local.macro, ~/.macro/global.macro ]
) ;

# Create a macro object and create initial macros/scripts from the
# definition(s) given:
my $Macro = Text::MacroScript->new(
-macro => [
[ MAX_INT => 32767 ],
],
-script => [
[ DHM2S =>
[
my $s = (#0*24*60*60)+(#1*60*60)+(#2*60) ;
"#0 days, #1 hrs, #2 mins = $s secs"
],
],
-variable => [ *MARKER* => 0 ],
) ;

# We may of course use any combination of the options.

my $Macro = Text::MacroScript->new( -comment => 1 ) ; # Create the %%[] macro.

# define()

$Macro->define( -macro, $macroname, $macrobody ) ;

$Macro->define( -script, $scriptname, $scriptbody ) ;

$Macro->define( -variable, $variablename, $variablebody ) ;

# undefine()

$Macro->undefine( -macro, $macroname ) ;

$Macro->undefine( -script, $scriptname ) ;

$Macro->undefine( -variable, $variablename ) ;

# undefine_all()

$Macro->undefine( -macro ) ;

$Macro->undefine( -script ) ;

$Macro->undefine( -variable ) ;

# list()

@macros = $Macro->list( -macro ) ;
@macros = $Macro->list( -macro, -namesonly ) ;

@scripts = $Macro->list( -script ) ;
@scripts = $Macro->list( -script, -namesonly ) ;

@variables = $Macro->list( -variable ) ;
@variables = $Macro->list( -variable, -namesonly ) ;

# load_file() - always treats the contents as within delimiters if we are
# doing embedded processing.

$Macro->load_file( $filename ) ;

# expand_file() - calls expand_embedded() if we are doing embedded
# processing otherwise calls expand().

$Macro->expand_file( $filename ) ;
@expanded = $Macro->expand_file( $filename ) ;


# expand()

$expanded = $Macro->expand( $unexpanded ) ;
$expanded = $Macro->expand( $unexpanded, $filename ) ;

# expand_embedded()

$expanded = $Macro->expand_embedded( $unexpanded ) ;
$expanded = $Macro->expand_embedded( $unexpanded, $filename ) ;

This bundle also includes the macro and macrodir scripts which allows us to expand macros without having to use/understand Text::MacroScript.pm, although you will have to learn the handful of macro commands available and which are documented here and in macro. macro provides more documentation on the embedded approach.

The macroutil.pl library supplied provides some functions which you may choose to use in HTML work for example.