Free search for common ground software for linux

CMU Common Lisp is a free implementation of the Common Lisp programming language which runs on most major Unix platforms. CMU Common Lisp project mainly conforms to the ANSI Common Lisp standard.
Main features:
- a sophisticated native-code compiler which is capable of powerful type inferences, and generates code competitive in speed with C compilers.
- generational garbage collection and multiprocessing capability on the x86 ports.
- a foreign function interface which allows interfacing with C code and system libraries, including shared libraries on most platforms, and direct access to Unix system calls.
- support for interprocess communication and remote procedure calls.
- an implementation of CLOS, the Common Lisp Object System, which includes multimethods and a metaobject protocol.
- a graphical source-level debugger using a Motif interface, and a code profiler.
- an interface to the X11 Window System (CLX), and a sophisticated graphical widget library (Garnet).
- programmer-extensible input and output streams.
- an Emacs-like editor implemented in Common Lisp.
- freely redistributable: free, with full source code (most of which is in the public domain) and no strings attached (and no warranty). Like the GNU/Linux and *BSD operating systems, CMUCL is maintained and improved by a team of volunteers collaborating over the Internet.
Common Lisp is well suited to large programming projects and explorative programming. The language has a dynamic semantics which distinguishes it from languages such as C and Ada.
It features automatic memory management, an interactive incremental development environment, a module system, a large number of powerful data structures, a large standard library of useful functions, a sophisticated object system supporting multiple inheritance and generic functions, an exception system, user-defined types and a macro system which allows programmers to extend the language.
Enhancements:
- A new float type EXT:DOUBLE-DOUBLE-FLOAT is supported.
- A DOUBLE-DOUBLE-FLOAT uses two DOUBLE-FLOATs to represent a number with >= 106 bits of precision (about 33 digits).
- Hash tables now support weak value, weak key- and-value, and weak key-or-value tables.
- LONG-LONG and UNSIGNED-LONG-LONG are recognized types in the C-CALL package for signed and unsigned 64-bit integers.
- The generational garbage collector has been ported to Darwin/ PPC.
- Numerous bugs and ANSI-compliance problems have been fixed.

Search::ContextGraph is a Perl module for spreading activation search engine.

SYNOPSIS

use Search::ContextGraph;

my $cg = Search::ContextGraph->new();

# first you add some documents, perhaps all at once...

my %docs = (
first => [ elephant, snake ],
second => [ camel, pony ],
third => { snake => 2, constrictor => 1 },
);

$cg->bulk_add( %docs );

# or in a loop...

foreach my $title ( keys %docs ) {
$cg->add( $title, $docs{$title} );
}

# or from a file...

my $cg = Search::ContextGraph->load_from_dir( "./myfiles" );

# you can store a graph object for later use

$cg->store( "stored.cng" );

# and retrieve it later...

my $cg = ContextGraph->retrieve( "stored.cng" );


# SEARCHING

# the easiest way

my @ranked_docs = $cg->simple_search( peanuts );


# get back both related terms and docs for more power

my ( $docs, $words ) = $cg->search(snake);


# you can use a document as your query

my ( $docs, $words ) = $cg->find_similar(First Document);


# Or you can query on a combination of things

my ( $docs, $words ) =
$cg->mixed_search( { docs => [ First Document ],
terms => [ snake, pony ]
);


# Print out result set of returned documents
foreach my $k ( sort { $docs->{$b} $docs->{$a} }
keys %{ $docs } ) {
print "Document $k had relevance ", $docs->{$k}, "n";
}

# Reload it
my $new = Search::ContextGraph->retrieve( "filename" );

Spreading activation is a neat technique for building search engines that return accurate results for a query even when there is no exact keyword match. The engine works by building a data structure called a context graph, which is a giant network of document and term nodes. All document nodes are connected to the terms that occur in that document; similarly, every term node is connected to all of the document nodes that term occurs in. We search the graph by starting at a query node and distributing a set amount of energy to its neighbor nodes. Then we recurse, diminishing the energy at each stage, until this spreading energy falls below a given threshold. Each node keeps track of accumulated energy, and this serves as our measure of relevance.

This means that documents that have many words in common will appear similar to the search engine. Likewise, words that occur together in many documents will be perceived as semantically related. Especially with larger, coherent document collections, the search engine can be quite effective at recognizing synonyms and finding useful relationships between documents. You can read a full description of the algorithm at http://www.nitle.org/papers/Contextual_Network_Graphs.pdf.

The search engine gives expanded recall (relevant results even when there is no keyword match) without incurring the kind of computational and patent issues posed by latent semantic indexing (LSI). The technique used here was originally described in a 1981 dissertation by Scott Preece.

Search::Tools are tools for building search applications.

SYNOPSIS

use Search::Tools;

my $re = Search::Tools->regexp(query => the quick brown fox);

my $snipper = Search::Tools->snipper(query => $re);

my $hiliter = Search::Tools->hiliter(query => $re);

for my $result (@search_results)
{
print $hiliter->light( $snipper->snip( $result->summary ) );
}

Search::Tools is a set of utilities for building search applications. Rather than adhering to a particular search application, the goal of Search::Tools is to provide general-purpose methods for common search application features. Think of Search::Tools like a toolbox rather than a hammer.

Examples include:

Parsing search queries for the meaningful keywords
Rich regular expressions for locating keywords in the original indexed documents
Contextual snippets showing query keywords
Highlighting of keywords in context

Search::Tools is derived from some of the features in HTML::HiLiter and SWISH::HiLiter, but has been re-written with an eye to accomodating more general purpose features.

Search::Tools::XML are methods for playing nice with XML and HTML.

SYNOPSIS

use Search::Tools::XML;

my $class = Search::Tools::XML;

my $text = the "quick brown" fox;

my $xml = $class->start_tag(foo);

$xml .= $class->utf8_safe( $text );

$xml .= $class->end_tag(foo);

# $xml: the "quick brown" fox

$class->escape( $xml );

# $xml: the "quick brown" fox

$class->unescape( $xml );

# $xml: the "quick brown" fox

my $plain = $class->no_html( $xml );

# $plain eq $text

Search::Tools::XML provides utility methods for dealing with XML and HTML. There isnt really anything new here that CPAN doesnt provide via HTML::Entities or similar modules. The difference is convenience: the most common methods you need for search apps are in one place with no extra dependencies.

NOTE: To get full UTF-8 character set from chr() you must be using Perl >= 5.8. This affects things like the unescape* methods.

DocSet::Doc::Common is a Perl module with common functions used in DocSet::Doc subclasses.

Implements functions and bits of code which otherwise needed to be duplicated in many modules. These functions couldnt be put into the base class DocSet::Doc. Certainly we could devise one more subclassing level but for now this gentle mix of inheritance and inclusion is doing its job just fine.

METHODS

postprocess_ps_pdf

$self->postprocess_ps_pdf()

renders ps and pdf version of a the current doc
fetch_pdf_doc_ver

%pdf_data = %{ $self->fetch_pdf_doc_ver() }

search for a pdf version of the same document in the parallel tree (usually the dst_html tree) and copy it to the same dir as the html version. Later we link to it from the html version of the document if the pdf version is found in the same directory as the html one.

The function returns a reference to a hash with the keys: size -- for the size of the file and the location of the file relative to the current document (its in the same directory after all).

fetch_src_doc_ver
similar to fetch_pdf_doc_ver() but works with the source version of the document.

%src_data = %{ $self->fetch_src_doc_ver() }

fetch the source version of the same document in the parallel tree (usually the src tree) and copy it to the same dir as the html version. Later we link to it from the html version of the document if the source version is found in the same directory as the html one. Notice that we add a .orig extension, because otherwise the rendered version of the source document may have the same full name as the source file (e.g. if the source was .html and destination one is .html too).

The function returns a reference to a hash with the keys: size -- for the size of the source file and the location of the file relative to the current document (its in the same directory after all).

pod_pom_html_view_seq_link_transform_path

my $linked_doc_path =
$self->pod_pom_html_view_seq_link_transform_path($src_path)

this is an implementation of the view_seq_link_transform_path() callback used in Pod::POM::HTML::view_seq_link(), using the DocSet::Docs transform_src_doc() method over pre-scanned cache of the source documents the $src_path is resolved into the path in the generated docset. So for example a the resource devel::help in L pod_pom_html_anchor($title);

this is a common function that takes the $title Pod::POM object, converts it into a < a name > html anchor and returns it.

Smart Common Input Method platform is a development platform that significantly reduces the difficulty of input method development.
SCIM splits input method into three parts: FrontEnd, which handles user interface and communication with client applications, Server, which handles the key event to string conversion work, and BackEnd, which manages all of the Servers.
Enhancements:
- The implementation of scim::Socket was improved for better error handling.
- A high power consumption issue caused by the X11 frontend was fixed.

Web Search Aggregator (WSA) provides multiple Web searches from one place. It features multiple categories and the display of results from many sites with a single search.
Your last search and the state of the tabs are remembered. Tabs can be navigated with the keyboard.
Enhancements:
- Mouse navigation has been added (hold down the Ctrl key).
- History of searches has been added (+statistics).