NAME SWISH - Perl interface to the SWISH-E search engine. SYNOPSIS use SWISH; $sh = SWISH->connect('Fork', prog => '/usr/local/bin/swish-e', indexes => 'index.swish-e', results => sub { print $_[1]->as_string,"\n" }, ); die $SWISH::errstr unless $sh; $hits = $sh->query('metaname=(foo or bar)'); print $hits ? "Returned $hits documents\n" : 'failed query:' . $sh->errstr . "\n"; # Variations $sh = SWISH->connect('Fork', prog => '/usr/local/bin/swish-e', indexes => \@indexes, results => \&results, # callback headers => \&headers, maxhits => 200, timeout => 20, -e => undef, # add just a switch ); $sh = SWISH->connect('Library', %parameters ); $sh = SWISH->connect('Library', \%parameters ); $sh = SWISH->connect('Server', port => $port_number, host => $host_name, %parameters, ); $hits = $sh->query( $query_string ); $hits = $sh->query( query => $query_string ); $hits = $sh->query( query => $query_string, results => \&results, headers => \&headers, properties => [qw/title subject/], sortorder => 'subject', startnum => 100, maxhits => 1000, ); $error_msg = $sh->error unless $hits; # Unusual, but might want to use in your headers() callback. $sh->abort_query; @raw_results = $sh->raw_query( \%query_settings ); $r = $sh->index( '/path/to/config' ); $r = $sh->index( \%indexing_settings ); # If all config settings were stored in the index header $r = $sh->reindex; $header_array_ref = $sh->indexheaders; # returns words as swish sees them for indexing $search_words = $sh->swish_words( \$doc ); $stemmed = $sh->stem_word( $word ); $sh->disconnect; # or an alias: $sh->close; DESCRIPTION NOTE: This is alpha code and is not to be used in a production environment. Testing and feedback on using this module is gratefully appreciated. This module provides a standard interface to the SWISH-E search engine. With this interface your program can use SWISH-E in the standard forking/exec method, or with the SWISH-E C library routines, and, if ever developed, the SWISH-E server with only a small change. The idea is that you can change the way your program accesses a SWISH-E index without having to change your code. Much, that is. METHODS Most methods will take either a hash, or a reference to a hash as a named parameter list. Parameters set in the connect() method will be defaults, with parameters in other methods overriding the defaults. connect `$sh = SWISH->connect( $access_method, \%params );' The connect method uses the `$access_method' to initiate a connection with SWISH-E. What that means depends on the access method. The return value is an object used to access methods below, or undefined if failed. Errors may be retrieved with the package variable $SWISH::errstr. The SWISH module will load the driver for the type of access specified in the access method, if available, by loading the `SWISH::$access_method module'. Parameters are described below in PARAMETERS, but must include the path to the swish binary program if using the File access_method and index file(s). (index files?) query `$hits = $sh->query( query => $query, \%parameters );' The query method executes a query and returns the number of hits found. `$hits' is undefined if there is an error. The last error may be retrieved with `$sh->error'. query can be passed a single scalar as the search string, a hash, or a reference to a hash. Parameters passed override the defaults specified in the connect method. Examples: $hits = $sh->query( 'foo or bar' ); $hits = $sh->query( 'subject=(foo or bar)' ); $hits = $sh->query( query => 'foo or bar' ); $hits = $sh->query( %parameters ); $hits = $sh->query( \%parameters ); It is recommended to use a callback function to receive the search results. See `headers' below. raw_query A raw_query returns a list containing every output line from the query, including index header lines. This can generate a large list, so using `query' with a callback function is recommended. Example: @results = $sh->raw_query('foo'); indexheaders The indexheaders method accesses the headers from the last `query' call. Since SWISH may return more than one set of headers (e.g. when searching multiple indexes), this method returns a reference to an array of hashes. Example: foreach my $header_set ( @{$sh->indexheaders} ) { print "\nHeaders:\n"; print "$_:$header_set->{$_}\n" for sort keys %$header_set; } abort_query Calling $sh->abort_query within your callback handlers (`results' and `headers') will terminate the current request. You could also probably just die() and get the same results. index ** To Be Implemented ** The index method creates a swish index file. You may pass `index' either a path to a SWISH-E configuration file, or reference to a hash with the index parameters stored in name => value pairs. The parameters in the hash will be written to a temporary file before indexing in with the Fork method. If passing a reference to a hash, you may include a key tempfile that specifies the location of the temporary file. Otherwise, /tmp will be assumed. If a parameter is not passed it will look in the object for an attribute named indexparam reindex ** To Be Implemented? ** This is a wish list method. The idea is all the indexing parameters would be stored in the header on an index so to all one would need to do to reindex is call swish with the name of the index file. stem_word ** To Be Implemented ** stem_word returns the stem of the word passed. This may be left to a separate module, but could be require()d on the fly. The swish stemming routine is needed to highlight search terms when the index contains stemmed words. swish_words ** To Be Implemented? ** swish_words takes a scalar or a reference to a scalar and tokenizes the words as swish would do during indexing. The return value is a reference to an array where each element is a token. Each token is also a reference to an array where the first element is the word, and the second element is a flag indicating if this is an indexable word. Confused? This requires HTML::Parser (HTML::TokeParser?) to be installed. The point of this is for enable phrase highlighting. You can read your source and, if lucky, highlight phrase found in searches. Example: $words = $sh->swish_words( 'This is a phrase of words' ); 0 1 2345 6 7 89 10 $words->[0][0] is 'This' $words->[0][1] is 1 indicating that swish would have this indexed $words->[0][2] is 0 this is swish word zero $words->[0][3] is the stemmed version of 'This', if using stemming. $words->[1][0] is ' ' $words->[1][1] is 0 indicating that swish would not index $words->[1][2] is undef (not a word) $words->[2][0] is 'is' $words->[2][1] is 0 indicating that swish would not index (stop word) $words->[2][2] is undef (not a word) $words->[6][0] is 'phrase' $words->[6][1] is 1 indicating that it is a swish word $words->[6][2] is 2 this is the second swish word ('is' and 'a' are stop words) ACCESS METHODS Two access methods are available: `Fork' and `Library'. The Fork method requires a `prog' parameter passed to the `connect' class method. This parameter specifies the location of the swish-e executable program The Library method does not require any special parameters, but does require that the SWISH::Library module is installed and can be found within @INC. The Server method is a proposed method to access a SWISH-E server. Required parameters may include `port', `host', `user', and `password' to gain access to the SWISH-E server. PARAMETERS Parameters can be specified when starting a swish connection. The parameters are stored as defaults within the object and will be used on each query, unless other overriding parameters are specified in an individual method call. Most parameters have been given longer names (below). But, any valid parameter may be specified by using the standard dash followed by a letter. That is: maxhits => 100, is the same as -m => 100, And to add just a switch without a parameter: -e => undef, Keep in mind that not all switches may work with all access methods. The swish binary may have different options than the swish library. prog prog defines the path to the swish executable. This is only used in the Fork access method. Example: $parameters{ path } = '/usr/local/bin/swish-e'; indexes indexes defines the index files used in the next query or raw_query operation. Examples: $parameters{ indexes } = '/path/to/index.swish-e'; $parameters{ indexes } = ['/path/to/index.swish-e', '/another/index']; query query defines the search words (-w switch for SWISH-E) Example: $parameters{ query } = 'keywords=(apples or oranges) and subject=(trees)'; tags or context tags (or the alias context) is a string that defines where to look in a HTML document (-t switch) properties properties defines which properties to return in the search results. Properties must be defined during indexing. You must pass an array reference if using more than one property. Examples: $sh = query( query => 'foo', properties => 'title' ); $sh = query( query => 'foo', properties => [qw/title subject/] ); maxhits Define the maximum number of results to return. Currently, If you specify more than one index file maxhits is per index file. startnum Defines the starting number in the results. This is used for generating paged results. Should there be pagesize and pagenum parameters? sortorder Sorts the results based on properties listed. Properties must be defined during indexing. You may specify ascending or descending sorts in future version of swish. Example: $parameters{ sortorder } = 'subject'; # under developement $parameters{ sortorder } = [qw/subject category/]; $parameters{ sortorder } = [qw/subject asc category desc/]; start_date ** Not implemented ** Specify a starting dates in unix seconds. Only results after this date will be returned. end_date ** Not implemented ** Ending date in unix seconds. results results defines a callback subroutine. This routine is called for each result returned by a query. Example: $parameters{ results } = \&display_results $parameters{ results } = sub { print $_[1]->file, "\n" }; Two paramaters are passed: the current search object (created by `connect') and an object blessed into the SWISH::Results class. Example: sub display_results { my ($sh, $hit) = @_; # SWISH::Results attributes my @show = qw/score file title size position total_hits/; my %results = map { ($_, $hit->$_) } @show; my @properties = @{$hit->{properties}} if $hit->{properties}; print join( ':', @results{ @show }, @properties ), "\n"; } The callback routines (`results' and `headers') are called while inside an eval block. If you die within your handlers the program will NOT exit, but any message you pass to die() will be available in $sh->errstr. In general, do as little as possible with your callback routines. The SWISH::Results class is currently within the SWISH module. This may change. headers headers defines a callback subroutine. This routine is called for each result returned by a query. Example: $parameters{ headers } = \&headers; Your subroutine is called with three parameters: the current object, and the header and value. sub headers { my ( $sh, $header, $value ) = @_; print "$header: $value\n"; } In general, it will be better to call the `headers' method. timeout timeout is the number of seconds to wait before aborting a query request. Don't spend too much time in your results callback routine if you are using a timeout. Timeout is emplemented as a $SIG{ALRM} handler and funny things happen with perl's signal handlers. TO DO How to detect a new index if library holds the file open? Is it ok to change index files on the same object? Does the library keep the index file open between requests? Interface for Windows platform? SEE ALSO http://sunsite.berkeley.edu/SWISH-E/ AUTHOR Bill Moseley