Thinking Sphinx on Windows

November 2nd, 2008

Thinking Sphinx on Windows

Tags: full-text search, rails, thinking sphinx

15 comments

Sphinx In one of my Ruby on Rails projects, I needed a full-text search engine to integrate. And after some research, I decided that Thinking Sphinx is the way to go. It seemed simple, fast and well documented. Unfortunately, I had some issues when tried to get it running on Windows.

The first step is installing Sphinx itself. While this looks simple, it took me five hours to move to the next step!

When I checked Sphinx documentation, it said that I needed Microsoft Visual C/C++ Studio to compile the source code! What? This is another project. After some (a lot of) research, some people suggested that I just needed to download the binaries, not the source code, and add the bin folder to the Windows Path environemnt variable. I tried that, but got errors later when tried to index data. So here is what works:

Go to Sphinx downloads page and download the Win32 release binaries with MySQL support. At the time of writing it is sphinx-0.9.8-win32.zip. Of course, you should get the one with PostgreSQL support if you need it.
Unzip the downloaded file and open the bin folder. Select and copy all files and paste them in your ruby/bin folder. If some files are already exist in your ruby/bin folder, don't replace them.
Install the Thinking Sphinx plugin:

ruby script/plugin install git://github.com/freelancing-god/thinking-sphinx.git

To set a model to be indexed, add some fields and attributes (refer to the primer):
```
class Post 
```
Tell Sphinx to index the data:
```
rake thinking_sphinx:index
```
Now, you may get something like this in your console:
```
rake thinking_sphinx:index
Generating Configuration to F:/InstantRails/rails_apps/community/config/development.sphinx.conf
indexer --config F:/InstantRails/rails_apps/community/config/development.sphinx.conf --all
```
It seems that it couldn't continue. To solve this, make sure you copied all the .exe files from sphinx-0.9.8-win32/bin to ruby/bin. If you keep getting the same result, you may need to copy the libmySQL.dll file from mysql/bin to ruby/bin (check this post for details). Now, you should get something like this:
```
Generating Configuration to F:/InstantRails/rails_apps/community/config/development.sphinx.conf
indexer --config F:/InstantRails/rails_apps/community/config/development.sphinx.conf --all
Sphinx 0.9.8-release (r1371)
Copyright (c) 2001-2008, Andrew Aksyonoff

using config file 'F:/InstantRails/rails_apps/community/config/development.sphinx.conf'...
indexing index 'post_core'...
collected 2 docs, 0.0 MB
collected 0 attr values
sorted 0.0 Mvalues, 100.0% done
sorted 0.0 Mhits, 100.0% done
total 2 docs, 27 bytes
total 0.030 sec, 904.37 bytes/sec, 66.99 docs/sec
distributed index 'post' can not be directly indexed; skipping.
```
Also, note the newly generated files:
- config/development.sphinx.conf
- db/sphinx/development/post_core.spa
- db/sphinx/development/post_core.spd
- db/sphinx/development/post_core.sph
- db/sphinx/development/post_core.spi
- db/sphinx/development/post_core.spm
- db/sphinx/development/post_core.spp
Great! But what was that last line about? Don't worry, Pat says it's OK:
A distributed index is made up of other indexes. It doesn't need to be indexed, but Sphinx tries to anyway, and then complains when it can't. I'm not sure about the reasoning for this, but it's not important: You can ignore the error, it doesn't stop anything working, searching and indexing will happen as expected.

Start a Sphinx searchd daemon:

rake thinking_sphinx:start

You should get something like this:

searchd --config F:/InstantRails/rails_apps/community/config/development.sphinx.conf
[Wed Oct 29 14:26:44.390 2008] [ 2796] WARNING: forcing --console mode on Windows
[Wed Oct 29 14:26:44.390 2008] [ 2796] using config file 'F:/InstantRails/rails_apps/community/config/development.sphinx.conf'...
[Wed Oct 29 14:26:44.406 2008] [ 2796] creating server socket on 127.0.0.1:3312
[Wed Oct 29 14:26:44.406 2008] [ 2796] accepting connections

Now, you are ready to search. To test it, you can modify the index action as follows:

class PostsController

And your view may look like this:

<% for post in @posts %>
  <h2><%=h post.subject %></h2>
  <%=h post.body %>
<% end %>

Now, navigate to /posts?search=myword where myword is a string that can be found in the title of some of your posts. You should see that the results are filtered according to the keyword sent.

Congratulations! You are thinking sphinx on Windows.

Did you like this article? Bookmark it:

15 Comments

jc
November 29th, 2008 - 09:10 PM

just what i was looking for! this should be on the thinking sphinx man page
farmer
December 15th, 2008 - 07:25 AM

Thanks, that was very helpful! There was one file in overlap - iconv.dll - as you mentioned might be the case. I just copied the .exe files and it fixed it. Also I first tried including the /bin directory for sphinx in my path but that didn't help.
Hatem
December 15th, 2008 - 03:38 PM

@jc: Thank you. I hope I saved you some time.
@farmer: Thanks, the same happened with me.
Praneeth Patlola
February 13th, 2009 - 01:55 PM

the command "ruby script/plugin install git://github.com/freelancing-god/thinking-sphinx.git" doesn't work. Can you please let me know if i were missing something. I do not have a folder or file called script under c:\ruby. Does this work only with a specific version
Aninda
February 15th, 2009 - 11:29 AM

Hi, Im facing a problem starting or indexing ts and I was wondering if this is a common problem. I get the following error when I try to run the rake or index: D:\Projects\RentFeedback>rake ts:in (in D:/Projects/RentFeedback) Generating Configuration to D:/Projects/RentFeedback/config/development.sphinx.conf C:/Ruby/bin/rake: No such file or directory - cat D:/Projects/RentFeedback/log/searchd.development.pid rake aborted! You have a nil object when you didn't expect it! You might have expected an instance of ActiveRecord::Base. The error occurred while evaluating nil.[] (See full trace by running task with --trace) The weird part is the rake and TS was running fine a day before. Any help appreciated, thanks.
Maxim
April 18th, 2009 - 06:20 PM

Thanks a lot for this guide! Been stuck with getting sphinx to run properly, but now it's working.
Andrew Chung
May 20th, 2009 - 09:12 PM

Instead of copying the .exe files to the ruby\bin directory, it may be easier to add the C:\Sphinx\bin directory to your PATH environment variable.
Ajay
July 10th, 2009 - 07:51 PM

Just wondering how to then get thinking sphinx to index the data hourly. I tried using task schedular but obviously with it running it can't write to the same file. Any help much appreciated
Ajay
July 10th, 2009 - 07:51 PM

Just wondering how to then get thinking sphinx to index the data hourly. I tried using task schedular but obviously with it running it can't write to the same file. Any help much appreciated
Hatem
July 13th, 2009 - 02:37 PM

@Andrew I have tried that first and it just didn't work.
Hatem
July 13th, 2009 - 06:00 PM

@Ajay you should use a delta index (Delayed Deltas): http://freelancing-god.github.com/ts/en/deltas.html
James
August 14th, 2009 - 02:55 PM

Great stuff, works like a charm! Thanks for this article!
sachin
September 13th, 2009 - 10:42 PM

nice job man it really helpful, first i tried to setup the latest version of sphinx it didn't work for me after i tried the version you mentioned in the post and its works fine, can you look what setting we have to change if we wants to run the rails application on the latest version of sphinx
Hatem
September 26th, 2009 - 04:24 PM

Sorry @sachin, I'm a little busy with other projects. May be soon.
Mohammed Aslam
December 23rd, 2009 - 05:18 PM

Hi, followed this article and when doing rake:thinking_sphinx:index i am getting the following error. Please help: C:\bc>rake thinking_sphinx:index (in C:/bc) Generating Configuration to C:/bc/config/development.sphinx.conf Sphinx 0.9.9-release (r2117) Copyright (c) 2001-2009, Andrew Aksyonoff using config file 'C:/bc/config/development.sphinx.conf'... FATAL: no indexes found in config file 'C:/bc/config/development.sphinx.conf'

ExpressionLab

Web Design

Web Development

Web Marketing