Rubicon 2 FAQ
Frequently Asked Question
Does Rubicon include source code? Yes. Source code is included for all units except five low level SysTools units licensed from TurboPower which we can only redistribute in DCU form. Are there any license fees or royalties on programs created with Rubicon? No. Does the download only version of Rubicon contain an electronic version of the manual? The files \rubicon2\doc\rubicon2.doc and \rubicon2\help\rubicon2.hlp contain all the information in the printed manual. Can multiple tables be indexed? Yes, but each table has its own Words table. Searches are usually specific to an individual table, but multiple tables may be searched with the TrbController component. Does Rubicon handle memo fields? Yes, Rubicon handles all standard field types. Memo fields are limited to 64k in 16 bit applications. Nonstandard fields such as ftBlob, ftFmtMemo, ftVarBytes, and 16 bit memo fields exceeding 64k can be handled via the OnProcessField event handler.Can Rubicon index Unicode character sets? No, Rubicon only works with ASCII character sets. Can Rubicon index blobs containing RTF? Yes. You will need to define an OnProcessField event in the TextLink like: procedure
TForm1.CustomProcessField(Sender: TObject; The dbRTFtoList procedure is defined in the rbRTF unit, so this unit will have to be added to the forms uses clause. Once defined, this procedure must be assigned to the OnProcessField event of each TextLink that accesses the table (i.e. when making, updating, or searching). How can I change the Rubicon a DataSet's configuration at run time? Nearly all Rubicon properties can safely be changed at run time, but there are a few things to look out for if you are changing a DataSet's properties or reassigning a TextLink or WordsLink. You should generally not directly open or close a DataSet (e.g. Table1.Close) that is connected to a WordsLink or TextLink. Instead, call the TextLink's or WordsLink's Open or Close method (e.g. rbTextTableLink1.Open). This will ensure that Rubicon knows that the state of the Dataset has changed. Finally, after you have made all your changes, call the engine's Reset method (see description in the addendum). Upgrading from Rubicon 1.x Trouble installing the components If you have installed a prior or trial run version of Rubicon, make sure you have deleted all the old files, especially the dcu/dpl/bpl files. You may want to confirm this by using the Windows 9x or NT 4.0 find utility and search for Rubicon files. Package files will not install There can be several causes for this. If you cannot install rb20rXXd or rb20bXXd (where XX is 30 for Delphi 3, 40 for Delphi 4, 50 for Delphi 5, etc.), then you will probably need to update Delphi to the latest version (see next question). If you are unable to install the drivers for a third party database engine, first confirm that the third party engine has been installed. If the problem persists, check the version of the third party engine to the version listed in c:\rubicon2\doc\drivers.htm. If they are different (and in most cases this is the cause), then the Rubicon drivers for that engine needs to be recompiled (see below). This usually does not require that the third party engine itself be recompiled nor does it usually mean that the core Rubicon components need recompiling (rb20rXXd or rb20bXXd where XX is 30 for Delphi 3, 40 for Delphi 4, 50 for Delphi 5, etc.). Which versions of Delphi are the package files compiled with? The precompiled packages included in the installation app are always compiled with the latest versions of Delphi available at the time of the release. For Delphi 3 version 3.02 is used and Delphi 4 Update #3 will be used as of Rubicon version 2.03. If you are using an earlier version of Delphi, you will have to recompile the packages (see next question). How can I recompile the packages? It shouldn't be necessary to recompile the packages unless you have run into a problem like the ones described above. The packages can generally be recompiled in the IDE, but the order in which they are recompiled is important. The run time packages should be compiled before the design time packages, and the core Rubicon components (rb20rXXr.dpk and rb20rXXd.dpk where XX is 30 for Delphi 3, 40 for Delphi 4, 50 for Delphi 5, etc.) should be recompiled before any drivers (if you can install the core Rubicon components, these packages don't need to be recompiled). If you encounter an error while recompiling the packages (e.g. internal error L552), try pressing the Compile button again. If the problem persists, try recompiling with the command line compiler (see below). After recompiling the packages, copy the run time dpl/bpl file(s) to either \windows\system or \winnt\system32. The run time package names end with "r". Common problems when recompiling The most common problem encountered when recompiling the packages are out of date dpl, bpl, or dcp files that are visible to the compiler. When searching for these files, be sure to check all your local hard disks. Pay special attention to your system directory, Rubicon directory (registered and trial run, if any), and the Delphi compiler Bin and Lib directories. For instance, if you get an error compiling rb20r40r.dpk (e.g. rb20r40r.dpk(33): Packages 'S401_R40' and 'rb20r40r' both contain unit 'STDict'), check your hard disk(s) for out of date rb20r40r.bpl and rb20r40r.dcp files and remove them. Under C++ Builder, also check for out of date bpi, obj, tds, and lib files. Recompiling packages with the command line compiler It is usually not necessary to use the command line compiler unless you are writing a batch file that automatically compiles a number of packages. The exception to this is when you run into an internal compiler error in the Delphi IDE (e.g. L552). In this case, you will have to use the command line compiler. The command line compiler is located in the Delphi \bin directory and is named dcc32.exe. Open a DOS window, change the directory to c:\rubicon2\d5 (or other appropriate subdirectory) and enter c:\delphi5\bin\dcc32 <rubicon package>.dpk -u<path names> where <rubicon package> is rb20f50r and <path names> are the paths (separated by semicolons) of any third party VCLs. For example: c:\delphi5\bin\dcc32 rb20f50r.dpk -uc:\FlashFiler For a complete listing of options, enter c:\delphi5\bin\dcc32.exe /? Remember to copy the run-time packages to your system directory after compiling them. These examples assume Delphi 5 is installed in c:\delphi5. The default installation is usually to c:\Program Files\Borland\Delphi5. If the command line compiler generates an internal error, then try deleting any old Rubicon bpl, dcp, and dof files. Installing Rubicon 2 with C++ Builder 4 and 5 The installation instructions in the \rubicon2\readme.txt file supercede those in the printed manual (the on disk manual and help files are up-to-date). Please refer to the readme.txt file for details. Installing with TurboPower's SysTools (Delphi 3 and above, C++ Builder 4 and above) If you own TurboPower products, you may have to add to the
Rubicon packages one of the following TurboPower packages: s401_r30,
s401_r35, s401_r40,
s401_r50, or s401_r60. Normally, a conflict will only occur when the SysTools package is
installed on the component palette. The error message you are likely to get is: If a conflict does occur, you will not be able to use the precompiled packages distributed with Rubicon. The alternatives include:
To add SysTools to rb20r60r, first open
rbDefine.inc and enable the $HaveSysTools compiler directive. Before
recompiling, delete any rb20*.?pl files on your system (check your system
directory too). Next, open rb20r60r.dpk in the Delphi IDE and add s401_r60 to the required package list and recompile the package. Then recompile and install the design time package
rb20r60d.dpk.
Repeat these steps for all the Rubicon packages installed in the IDE. After recompiling the
packages, exit Delphi and copy the run time bpl files (rb20?60r.bpl) to your
system directory (\windows\system, \winnt\system32, or \windows\system32).
When you restart Delphi, all the Rubicon packages should successfully load. After installing Rubicon, I get an error when I restart Delphi The most common cause for this is recompiling the packages, but forgetting to copy the run time packages to \windows\system or \winnt\system32.
Is Opus DirectAccess supported? Rubicon is compatible with Opus DirectAccess. To use Rubicon with ODA, the UseODA compiler directive must be enabled in rbDefine.inc and the Rubicon packages recompiled under ODA. Does the TQuery driver support Oracle? The TQuery driver has been tested with Oracle 8. The only restriction is that you have to enter table names and field names in uppercase. How should the datasets be configured? The datasets used by Rubicon to access the Text and Words
(via the TextLink and WordsLink
components) should only have their essential properties set (e.g. DatabaseName,
TableName, in most cases Active,
etc.). For some datasets, this would also include setting of connections, transactions,
etc. When using the query based links, what should be included in the SQL property? Nothing. Rubicon will manage the SQL property. Can the same cache be used to cache indexes from different Words tables? No, the TrbCache component can only cache indexes from one Words table. How can I expand acronyms? Use the OnProcessField event. Can I reduce the size of Words table by setting WordFieldSize to a lower value? Yes, but you run the risk of increasing the number of truncated words, which leads to false matches during updates and ambiguous search results. Does the AltMemMgr option replace the standard Delphi memory manager? No, it merely supplements the standard memory manager during GetMem and FreeMem calls, and only is used for cache memory. The blob portion of the Words seems excessively large Check to see if the table type being used has a default or minimum blob size. If so, see if the default size can be reduced to 32 or 64 bytes. For dBase tables, the MEMO FILE BLOCK SIZE is often set to 1024 or 2048 in BDEAdmin.exe, which is excessively high. Can a Words table built with a 16 bit application be read by a 32 bit application? Yes, and visa versa. Can binary field (BytesFieldSize) be used with InterBase tables? No. Despite the fact that InterBase supports the ftBytes field type, testing shows that Rubicon is not compatible with the way this field type it has been implemented. Words seem to be missing or incorrectly associated in the dictionary If the length of the words in question exceeds the WordFieldSize property, increase WordFieldSize and rebuild the dictionary. MemoryUsage includes what kinds of memory?It is primarily made up of the memory used to cache the indexes. If AltMemMgr is True, it also includes any memory in the memory pool. Some internal buffers are also included. It does not include the memory used by the container classes themselves, various TLists, and other ancillary data structures. When using the BDE, which local table format is best for the Words table? Paradox since it natively supports the ftBytes field type (see BytesFieldSize) and has very little overhead for blob fields.Processing TrbMake.Execute slows down exponentiallyThere may be insufficient memory to complete the operation. Check the value of the caches MemoryLimit property. You may need to increase it from the default value of 4mb (advise setting it to the amount of physical memory installed minus 4 to 8 mb for Win9x, half of physical memory for NT). If using Delphi 2 or higher, you may have run into the memory fragmentation bug. Set the AltMemMgr property to True.Words at the end of memos are not indexed 16 bit applications are limited to memo lengths of 64kb. If possible, compile your application with Delphi 2 or higher. Number of Words table records varies with table type Normally, the number of unique words should not vary with table type. Differences can arise when the source table(s) contain nonstandard characters that are treated differently by the table types, and therefore result in key violations that cause a word to be excluded from the table. For instance, one table may interpret Canada and Cañada as two different words, the other may treat them as the same (and thus one would be excluded because of a key violation). Key Violations This problem is similar to the problem described above. If Rubicon indexes the words Canada and Cañada, but the database saves Cañada as Canada, then a key violation will occur. To test for this problem, add '<<<' and '>>>' to the WordDelims, and re-index. If you can re-index, then remove '>>>' from the WordDelims (it is probably safe to keep '<<<', see WordDelims in the help file for an explanation of what these strings do) and then change the Words table's language driver or collation to one that is compatible with the Text table's character set. All the values for WordCount and BlobSize are zero in my Words tableThis should only occur when using TrbCustomWordsBDELink components. This indicates that the database format of Words table does not support 32 bit integers. Check to be sure that the dbiWrite property is set to False and rebuild the Words table.Using Calculated or Lookup Fields with Query-like DataSets Unlike table-like datasets, Rubicon controls all the fields in a query-like dataset. This makes it impossible to use calculated or lookup fields the conventional way. To achieve the same result, use the OnProcessField event to perform the calculation or lookup and pass the results to the Rubicon engine for processing. procedure
TForm1.CustomProcessField(Sender: TObject; Remember, this method must be assigned to the OnProcessField event of each TextLink that accesses the table (i.e. when making, updating, or searching). Does the Words table have to be recreated when rebuilding the Words table? The default behavior of Rubicon is to create a new Words table each time TrbMake.Execute is called. The primary reason for this is that this ensures that the field definitions in the Words table match the properties in TrbWordsDataSetLink (e.g. WordFieldSize). Deleting an existing Words table and recreating it is usually the fastest way to "empty" the table. However, in some cases this behavior may be undesirable given the user's database rights. If the Words table already exists, no properties in the TrbWordsDataSetLink are changing, and TrbMake.SegmentSize is not changing, then the existing Words table can be reused by performing these steps:
I get a "Dictionary invalidated" error when performing an update If the application is inserting records before the first record (in IndexFieldName order) or deleting the first record, this error will be raised. To fix the problem, an OnMinIndex event will have to be added and Words table rebuilt. Once an OnMinIndex event is used, it will also have to be assigned to any update or search component used on the Text table. The error can also occur when an update is performed and a word that should be found in the Words table is not found, this error is raised. In this case, the Words table has become corrupted and must be rebuilt. This is usually caused when the parsing or testing of words with TrbAccept is setup differently for making and updating the Words table. For example, a word rejected during a make is accepted during an update. The update component expects to find the word, but when it does not find it, it raises the error. Finally, for the BDE drivers, this error is also raised for Paradox and dBase tables when inserts in the front or middle of the table are performed and the IndexFieldName is blank. This is a limitation imposed by not using an IndexFieldName. Performing Updates with Queries My application seems to stall while using TrbUpdate. Can this be avoided? If you have set DelayedWrites to True, TrbUpdate will write records to disk when the cache is full. You may use the WordsLInk AfterPost event to do some processing while the cache is being compressed. You should not interrupt this process. Calls to WriteCache or FlushCache may also cause delays. Here, you may abort the process and then resume it later. Can TrbUpdate be used against a newly created table? Generally, no. TrbUpdate is designed to update indexes in an existing Words table, not to create and populate a new Words table. Instead, use TrbMake to create the initial Words table, then use TrbUpdate to keep it current. The first update takes longer than subsequent updates See below "The first search takes longer than subsequent searches". Is there an event or message that I can use to indicate that the update has been completed so that I can safely perform a search? There is not an event, but a search may be safely performed under the following conditions:
The first update takes longer than subsequent updates When the first update is executed, TrbUpdate needs to initialize itself as well as perform the update. You may minimize the impact of this by performing the initialization earlier by calling Initialize. If you are working with SQL tables, you will want to read the section in the manual "Working with SQL Tables" or see the OnMinIndex and OnMaxIndex events in the help file.How can I use TrbSearch as a filter for my DataSet? In Delphi 2.0 or higher and C++ Builder, you may simply define an OnFilterRecord event handler and test whether the current record matches the search criteria by calling rbSearch1.Matches. In Delphi 1.0, you will have to define a dbiAddFilter routine or equivalent. The first search takes longer than subsequent searches When the first search is executed, TrbSearch needs to initialize itself as well as perform the search. You may minimize the impact of this by performing the initialization earlier by calling Initialize. If you are working with SQL tables, you will want to read the section in the manual "Working with SQL Tables" or see the OnMinIndex and OnMaxIndex events in the help file. Search returns no matching records Assuming that the search should return matching records, the most likely cause for this is that one or more words in the search have not been indexed. To confirm this, check the Words table to see if the words are missing. If so, then check whether the missing word is being rejected because of MinWordLen, WordDelims, or the OnAcceptWord event. Searches are not finding the correct records If a TrbTextBDELink is being used with a blank IndexFieldName, then a record may have been inserted or deleted in the middle of the table. This will corrupt all the Words table index values and does not generate an error unless the change was made when TrbUpdate was running. Otherwise, check for table corruption and use the Verify utility to check the Words table. A slOr search on * followed by a slNot search should return zero matches, but doesnt What is being returned are the gaps between index values. Since these records dont really exist, a call to TrbMatchMaker.Execute will return an empty table. The correct way to perform the above search is to follow the slOr search followed by a slNot NarrowSearch. The Matches method does not seem to be working Matches returns a value that indicates whether the current record in the DataSet meets the search criteria. You may have to call UpdateCursorPos before calling Matches. In addition, when using a TrbTextBDELink with a blank IndexFieldName and Matches is called from within a filter, it may not be possible to synchronize the DataSet to the physical record number. What is the state of the search if the TimeLimit has been exceeded? The search results are incomplete. Usually a search will only timeout during wildcard and proximity searches. When a wildcard search is interrupted, not all of the matching locations have been found. When a proximity search has been interrupted, not all of the matching locations have been checked to determine whether they meet the proximity criteria. Those locations not checked are removed from the search results. If the TimeLimit is exceeded, the OnTimeout event is called. procedure
TForm1.rbSearch1Timeout(Sender: TObject); Please note that during the search, SearchTime is the tick count at the beginning of the search. After the search has been completed, SearchTime is the elapsed time of the search. If you use the above code, remember to reset the TimeLimit before the next search! Does the Match dataset have to be the same table type as the table being searched? No. In a client/server situation, you may not want to create a Match dataset on the server and instead create it on the client side using a local table type or an in-memory table such as a TClientDataSet. Can the Match dataset be a TQuery? No. TrbMatchMaker does not support query-like datasets. In C/S multi-user environment where this Match dataset created? It depends on the table type of the Match dataset. If it is an SQL table, then it would reside on the server. Is it possible to create the Match dataset on the client side? Yes, use a local table such as Paradox or use an in-memory dataset such as a TClientDataSet. Some of the examples use a TClientDataSet which I do not have. What can I do? The TClientDataSet component is included with the Client/Server or Enterprise versions of Delphi and C++ Builder. An alternative to the TClientDataSet is the TMemDataSet component (see www.cam.org/~abrunet/index.html). In order to use TMemDataSet, the rbMemDS unit must be included in the USES clause. Does TrbMatchMaker creates separate Match dataset for each user?This depends on the type of Match dataset being used. A TClientDataSet would be for each user. For an SQL table, it would be on the server, so each user would have to be assigned a unique table name by your application.Can the process of creating the Match dataset be interrupted by user if it takes long time? Yes. Use the TrbMatchMaker MatchLimit or TimeLimit properties. The process may also be aborted inside the OnAcceptRecord event:function
TForm1.rbMatchMaker1AcceptRecord(Sender: TObject): Boolean; How can I create a Match table with DBISAM 1.xx? DBISAM 1.xx requires that tables have a primary index, so an index has to be added to the Match table definition before it is created. For more information, see the comments in the drivers page or see c:\rubicon2\examples\dbisam\ExSrch.dpr. Note this does not apply to DBISAM 2.xx. A match table record has a rank of zero. If RankMode is rmPercent, it is possible that the rank value is being rounded to zero. This may be checked by changing the RankMode to rmCount and rerun the search.Also check to see if the word or words that should have been ranked have a length that exceeds the WordFieldSize property. Typically this occurs when using a wildcard in a search. For instance, a search for WaitFor* might locate a record containing WaitForMultipleObjects, but the word found in the index and listed in the MatchingWords property is WaitForMultipleObjects. The solution here is to increase the value of WordFieldSize.The MatchCount is correct but I cannot see the result table because it is always empty First check to make sure that the rbMatchMaker1 DataSet and Searcher properties are set correctly. Next check the field type of the IndexFieldName field. If it is a string field and it contains leading zeros (e.g. 001234), try removing the leading zeros or convert the field type to integer (recommended). What does the "Decompress buffer too small" error mean? The error means that there was not enough memory allocated to decompress an index. The allocation of this memory is handled internally and is not affected by MemoryLimit or the amount of installed memory on the computer. This condition is usually caused by one of the following: When updates and searches are occurring simultaneously, the search application may not have allocated enough memory to hold the word indexes (which may have grown in size). In this case, see the code in Button1Click in c:\rubicon2\examples\ttable\ExSrch.dpr which handles this exception. If the application is placing ranges or filters on the Text table, this will cause the error. In this case, call rbUpdate1.Initialize or rbSearch1.Initialize before setting any filters or ranges. Also, read the "Filters and Ranges" section in the documentation. Finally, this may result from records being deleted from the Text table without Rubicon being notified. Rebuilding the Words table should solve the problem. I get a "Dictionary invalidated" error when performing an update See answer in the "Updating Indexes" section. A "Dataset not supported" error is raised when using a TrbMatchMaker If you are using a TClientDataSet or a TMemDataSet, then you need to include rbCDS or rbMemDS, respectively, in the uses statement. For other datasets, include the appropriate Rubicon unit in the uses statement (e.g. for a Paradox Match table, include rbTable in the uses statement). SQL based datasets are generally not supported because of multi-user and efficiency issues.A "Class not registered" error is raised when using a TClientDataSet Check that DBCLIENT.DLL is in the system directory and that it is up-to-date. I get the error message "Field rbWord must have a value" Usually it is an odd ASCII character that has crept in to a field. To test for this, add '<<<' to the WordDelims (without the quotes) -- this will force #0..#31 to be treated as WordDelims. You may also try adding '>>>' to the WordDelims -- this will treat #128..#255 as WordDelims. The language driver being used may also be part of the problem. I get the error message "Invalid Block Size" during a make or update This is a rare error and, based on past experience, is usually due to a memory overwrite somewhere in your code. If you are doing any processing of the record in an OnProcessField event, then this is the most likely suspect. If you don't think your code is causing the problem, try recreating the error with ExMake.dpr (pointing the Tables to your tables). What causes the error "Field rbWord must have a value"? Usually it is an odd ASCII character that has crept in to a field. To test for this, add '<<<' to the WordDelims (without the quotes) -- this will force #0..#31 to be treated as WordDelims. You may also try adding '>>>' to the WordDelims -- this will treat #128..#255 as WordDelims. Here is an example: when parsing the text, if Rubicon finds a word
consisting of the characters #200#200, then Rubicon will index the word
and try to write it to the Words table. The database engine may also
parse the word (often depending on the colation or language driver) and may
convert #200#200 to an empty string which will raise this error. By
adding #200 to the WordDelims, the
character #200 will not be indexed. Limitations and Known Problems
|
Copyright 2003 © Tamarack Associates |
||
www.TamarackA.com | Last updated 01/16/03 | www.FullTextSearch.com |