Sphider 1.5.0 Search Tool is now live

Find the new Sphider 1.5.0 on our Downloads page.

UPDATE: 1 December 2015,   18:25 UTC. If you downloaded 1.5.0 before this time, the auto-suggest may not work if you installed Sphider to any directory NOT named /sphider. The current posting DOES work.

If you are affected, you do not need to re-download. Simply find “autocomplete.js” in the js_suggest directory,and edit line 7 from:
$.get( “/sphider/js_suggest/suggest.php”, { keyword: keyword } )
$.get( “js_suggest/suggest.php”, { keyword: keyword } )

Football Bowls

Are you a college football fan? I used to be. That’s right, used to be! For one thing, college football has gotten completely out of hand. It is now driven by bucks, BIG bucks. Then there is the media hype. All in all, the hype, the show-boating, the scandals, the money… these things have driven me away in large part.

And now the time of year is coming when the plethora of bowls begin. I just read that this year there will be 40 different bowl games! Can you believe that? Forty bowl games, requiring 80 competing teams. It seems bowl officials are having difficulty finding enough teams with winning records to fill all the slots. This means that teams with losing seasons still have a shot at being in a bowl.

In 1970, there were a total of eleven bowls, four major, seven minor. In 1970, only the best of the best got to go to a bowl. This made the bowls meaningful, worth going to, or watching, or even listening on the radio.

Today, the thrill is gone. The major bowls, heck, all the bowls are more hype than substance. It’s all being driven by profit. Advertising.

For the sake of the game, I for one think it’s time to cut back on the number of bowl games. W-A-Y back! I’m thinking maybe 15 bowls, tops!

But then again, I do have my quirks.

This is why we decided to update Sphider!

85 CVE-2014-5194 94 1 2014-08-07 2014-08-07
None Remote Low Single system Partial Partial Partial
Static code injection vulnerability in admin/admin.php in Sphider 1.3.6 allows remote
authenticated users to inject arbitrary PHP code into settings/conf.php via
the _word_upper_bound parameter.
86 CVE-2014-5193 79 1 XSS 2014-08-07 2014-08-22
None Remote Medium Not required None Partial None
Cross-site scripting (XSS) vulnerability in admin/admin.php in Sphider 1.3.6 allows
remote attackers to inject arbitrary web script or HTML via the category parameter.
NOTE: the url parameter vector is already covered by CVE-2014-5082.
91 CVE-2014-5082 89 1 Exec Code Sql 2014-08-06 2015-11-04
None Remote Low Not required Partial Partial Partial
Multiple SQL injection vulnerabilities in admin/admin.php in Sphider 1.3.6 and
earlier, Sphider Pro, and Sphider-plus allow remote attackers to execute arbitrary
SQL commands via the (1) site_id or (2) url parameter.

And it all started because the deprecated code was making Sphider useless. The deeper we dug…

Ain’t that the way life works? You start out fixing one problem and find two more to take its place.

Have we fixed ALL the problems? Probably not, but it is a darn good start!

Sphider 1.5.0

Version 1.5.0 is just days away from public release. Testing is continuing, but going very well. The NEW Sphider User’s Guide is essentially complete and being reviewed. We could say it is ready to go now, but rather put it through its paces a couple more times before publishing it on Tuesday, December 1, 2015.

So what’s so special about Sphider 1.5.0? For starters, it is fully up to date. PHP 5.6 loves it. So does MySQL 5.6. And all the html is HTML5, which is very strict in structure. Because Sphider is written in PHP, if you do a “view source” on a web page it ain’t gonna be pretty. But it is correct.

Having read the change reports for MySQL 5.7, which was just recently released, no changes appear to be needed. And, although still in Release Candidate stage, early examination of the coming PHP 7 does not reveal any issues. Sphider 1.5.0 is ready!

Security has also been a concern. Sphider 1.3.6 is ripe with opportunity for SQL injection attacks. Simply getting rid of the deprecated MySQL extension and replacing it with the MySQLi extension did nothing for security. Moving to prepared statements DOES virtually eliminate SQL injection attacks. With prepared statements, bound variables are kept separate and never parsed as a part of a generic SQL statement.

Wherever GET, POST, or REQUEST data is used, it is escaped, matched, and any otherwise reduced to safe data.

One critical Sphider page was once COMPLETELY rewritten, using unescaped GET data everytime the settings were changed. No more. This page (which you never actually SEE), is now static in structure and completed on call from the database. Of course, changing the configuration means updating the database, which in turn uses GET data. The thing is, now 1) the GET data is parsed and escaped, and 2) is written to the database using the prepared statement process. This critical page can no longer be hijacked and used as a weapon against you.

Originally, there was some PHP code written into some .html pages. If you looked at the page in a browser and went to “view source”, anyone could view snippets of actual PHP code. No more.

We also found that, if you dug deep enough into the spider functions, our earlier efforts to improve Sphider broke a couple things. We corrected those. We ALSO found there were things in 1.3.6 that were SUPPOSED to work, but didn’t. We corrected those, too.

So, are we claiming Sphider 1.5.0 to be bug free, the perfect Sphider? No, we aren’t that vain. But, for today’s environment, Sphider 1.5.0 is a good fix for the dying Sphider 1.3.6. And you won’t have to pay money for Sphider-plus or Sphider-pro and get functionality you don’t need.

We feel the Sphider User’s Guide is l-o-n-g overdo. Wouldn’t it be nice to really know what all the setting do, what happens on each of the admin pages, and what kind of searches you can do? It’s always nice to have a road map.

(Did we mention that spelling suggestions now works much more reliably? I supposed it USED to work before, but in today’s world, it was a no-show on most browsers we tried.)

Watch our Downloads page. Sphider 1.5.0 should be making its appearance there this coming Tuesday.

Sphider Search Tool

A long time ago, WorldSpaceFlight had nothing but html pages. Indexing the site was a pretty easy task using a simple perl script, and a user search was pretty easy.

Time passed, and the first php pages were introduced, pages which could not be indexed by the perl script. What started as a minor problem with a few pages grew into a MAJOR problem with nearly ALL the pages as we progressed towards a complete (almost) use of php pages.

Then enter Sphider. This is a tool from www.sphider.eu which allowed php pages to be parsed an indexed. Things were great. But time passed, the state of PHP and MySQL advanced, but alas, Sphider did not. With only one security update, Sphider has remained the same for the last 6 years. Security problems, deprecated code, things that just plain no longer worked…

Sphider became useless. Looking for a suitable replacement proved fruitless.

So, we did what any practical person would. If the tool was broken, FIX IT! And we did, but just for ourselves. We had a Sphider that worked, but it was still a huge security risk, highly vulnerable to SQL injection or remote code execution attacks. We protected ourselves the best we could, but finally decided that just wasn’t enough!

The result was Sphider 1.4.1. Sphider was considerably hardened. But that wasn’t enough for us. We wanted more, and since we had already dug into Sphider’s code, we did more. We added a wildcard search and some new templates. The default “standard” was, yawn, boring! And the “dark” template was, well, dark! And since the PHP was up to date, we made all the HTML up to date, HTML5. We also made it possible to index pdf files in a Windows environment, and not just in Linux. We threw in a few bug fixes while we were at it. Thus was born Sphider 1.4.2.

Not satisfied with what we had, we have been working on yet ANOTHER update. The suggest function, which USED to work way back when, had become rather finicky, working for some browsers, not for other. So we reworked the who “suggest” feature. Then we converted all queries to use prepared statements. This virtually eliminates SQL injection attacks. More bug fixes, a couple enhanced features here and there. The code has been cleaned up considerably. This will be Sphider 1.5.0. It is a MAJOR change from 1.4.2. We are still testing, as we made SO MANY changes we want to be sure nothing got broken in the process. When it is ready, there will be a User’s Guide to go with it, something which has been lacking. Searching the forums can only go so far.