Sphider 3.1.1-MB corrects an indexing issue

The original Sphider was built with the English language in mind. While preparing to store indexed words, a check is performed to ensure the word is composed of alphabetic characters. Knowing that sites other than all English sites might be indexed, this check was modified to accommodate non-English but still Western characters. The function used for this was what eventually was named removeAccents(). It actually worked quite well. Sphider advanced and took on still more languages, particularly those with non-Western alphabets. Surprisingly, Sphider somehow kept chugging along, although just how is a bit of a mystery.

Well, recognizing some of the short comings of removeAccents(),  it was “improved” in Sphider 3.1.0. And it WAS improved… in some instances. Unfortunately, it was discovered (after it had been released!!  🙁  ), that Aribic words would no longer index. Other languages were probably similarly impacted, but just one is enough to raise red flags.

Rather than fix a patch that was originally intended to cover a minor flaw, it seemed logical that the initial check on the word should be updated to conform to the realities of unicode. The check to be sure the word is alphabetic now truly covers the many alphabets rather try to patch the check for an ASCII alphabet.

Sphider 3.1.0-MB and Sphider 2.4.2-PDO released

Sphider 3.1.0-MB is multibyte capable, like 3.0.0-MB. However, 3.1.0-MB does NOT require the PHP mbstring extension. Mbstring is recommended, but not required. If it is available, it will be used. If not, Sphider will emulate the mulitbyte character string functions. Also, 3.1.0-MB continues the improvements always being made to the original fork. Since there is no longer any special requirements other than the typical MySQLi/MySQLnd extensions, there is no longer a need for the 2.4.x line.

Sphider 2.4.2-PDO provides a fix for a problem with 2.4.1-PDO which could cause some UTF-8 characters to be mistaken for ISO-8859-1 characters. The resulting “conversion” produced rubbish. The PDO fork will continue to be available and supported, but no further product enhancements are anticipated.