Solarium 2.0

Several weeks ago Solarium 1.0 was released. Since then lots of development has been going on. Many features were added: MoreLikeThis support, range facet, multiQuery facet, DisMax support, geospatial search support and highlighting. The target for these features was originally Solarium 1.1, however I’ve changed the plans.
In this post I’ll explain why, and what the important changes in 2.0 will be.

What features would you like to get in Solarium?

Solarium is quite a young project, and there are still a lot of features to add. The project has been gaining some interest recently and I would really like to know which features are most wanted.

So, I’ve created a poll. The most requested features will be placed at the top of the roadmap.

Solarium PHP Solr client

I’ve worked on a lot of Solr implementations in PHP applications. There are multiple solutions: manual HTTP requests, the solr-php-client library, custom implementations etcetera. However they all have one issue in common: they only handle the communication with Solr, many other important parts like query building are not covered at all. And the parts that are covered are usually over-simplified.

In my previous post Integrating Solr with PHP I did a comparison of several of the available options. Since then I’ve done more research and started to make notes of all issues I came across and all features I missed. Based on these notes I’ve started a project that tries to accurately model Solr and go one step beyond the existing solutions.

Stunnel, a practical solution for SSL/TLS

TLS/SSL encryption is generaly a good thing to use, however it can in some cases be hard to implement. Stunnel is a small program that provides SSL/TLS capabilities to clients and/or servers that don’t speak SSL/TLS natively. It basically works like a proxy, transparently adding an encryption layer to the communication layer; or removing it.

There are two main use cases:

• connecting to an encrypted service using a plaintext client
• publish an encrypted service, while the actual backend is plaintext

There can be multiple reasons to use Stunnel instead of an implementation directly in the client or server application. You might be using an application that you cannot change. Or it might cost to much time. And Stunnel might in some cases even offer some features that are hard to achieve otherwise.

Integrating Solr with PHP

Currently there isn’t really a standard way to integrate Solr with PHP. For various projects I have used:

• manual HTTP requests
• PHP Solr extension
• solr-php-client library

They all have their own pros and cons.

Controlling Varnish ESI inside your application

For me one of the best features of Varnish is ESI (Edge Side Includes).  It allows you to combine elements with different lifetimes into a single page. This way you don’t need to regenerate a complete page as soon as a news listing somewhere on the page changes. And you can still cache pages while displaying user-specific information somewhere on the page.
This lowers the load on your application even further and it can also help in simplifying your application, by focussing purely on the content of the page and loading all other elements via ESI.

There is a small drawback though, parsing for ESI comes at a cost. Although Varnish performs well even with ESI enabled, the effect is noticable. And binary files should be excluded from ESI parsing all together. The usual solution is to enable ESI only for specific requests in your VCL file. A typical example:

sub vcl_fetch {
    if (req.url == "/news") {
        esi;  /* Do ESI processing */
    }
}

With this solution you might end up adding all URLs that need to be parsed into your VCL file, complicating application development. Or you can decide to parse all text/html content, so you don’t need to frequently alter your VCL. Both solutions work, but are not optimal. The main issue is that your application should decide when to apply ESI to the content, not Varnish. The knowledge to base this decision on is in your application domain. When developing a page in your application you know whether you want to use ESI, and you should be able to do so easily, right inside the application.