A long time ago, WorldSpaceFlight had nothing but html pages. Indexing the site was a pretty easy task using a simple perl script, and a user search was pretty easy.
Time passed, and the first php pages were introduced, pages which could not be indexed by the perl script. What started as a minor problem with a few pages grew into a MAJOR problem with nearly ALL the pages as we progressed towards a complete (almost) use of php pages.
Then enter Sphider. This is a tool from www.sphider.eu which allowed php pages to be parsed an indexed. Things were great. But time passed, the state of PHP and MySQL advanced, but alas, Sphider did not. With only one security update, Sphider has remained the same for the last 6 years. Security problems, deprecated code, things that just plain no longer worked…
Sphider became useless. Looking for a suitable replacement proved fruitless.
So, we did what any practical person would. If the tool was broken, FIX IT! And we did, but just for ourselves. We had a Sphider that worked, but it was still a huge security risk, highly vulnerable to SQL injection or remote code execution attacks. We protected ourselves the best we could, but finally decided that just wasn’t enough!
The result was Sphider 1.4.1. Sphider was considerably hardened. But that wasn’t enough for us. We wanted more, and since we had already dug into Sphider’s code, we did more. We added a wildcard search and some new templates. The default “standard” was, yawn, boring! And the “dark” template was, well, dark! And since the PHP was up to date, we made all the HTML up to date, HTML5. We also made it possible to index pdf files in a Windows environment, and not just in Linux. We threw in a few bug fixes while we were at it. Thus was born Sphider 1.4.2.
Not satisfied with what we had, we have been working on yet ANOTHER update. The suggest function, which USED to work way back when, had become rather finicky, working for some browsers, not for other. So we reworked the whole”suggest” feature. Then we converted all queries to use prepared statements. This virtually eliminates SQL injection attacks. More bug fixes, a couple enhanced features here and there. The code has been cleaned up considerably. This will be Sphider 1.5.0. It is a MAJOR change from 1.4.2. We are still testing, as we made SO MANY changes we want to be sure nothing got broken in the process. When it is ready, there will be a User’s Guide to go with it, something which has been lacking. Searching the forums can only go so far.