Buy

Designing how to Filter

Paginated a big collection is a must. But you might also want a client to be able to search or filter that collection. Ok, so how do we search on the web? Well usually, you fill in a box, hit submit, and that makes a GET request with your search term as a query parameter like ?q=. The server reads that and returns the results.

I have an idea! Let's do the exact same thing! First, we will of course add a test. Add a new programmer at the top of the pagination test with $this->createProgrammer(). I want to do a search that will not return this new programmer, but still will return the original 25. To do that, give it a totally different nickname, like 'nickname' => 'willnotmatch'. Keep the avatar number as 3... because we don't really care:

228 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
$this->createProgrammer(array(
'nickname' => 'willnotmatch',
'avatarNumber' => 5,
));
... lines 78 - 120
}
... lines 122 - 226
}

For the query parameter, use whatever name you want: how about ?filter=programmer:

228 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 85
// page 1
$response = $this->client->get('/api/programmers?filter=programmer');
... lines 88 - 228

If you're feeling fancy, you could have multiple query parameters for different fields, or some cool search syntax like on GitHub. That's all up to you - the API will still work exactly the same.

Filtering the Collection

Great news: it turns out that this is going to be pretty easy. First, get the filter value: $filter = $request->query->get('filter');. Pass that to the "query builder" function as an argument. Let's update that to handle a filter string:

191 lines src/AppBundle/Controller/Api/ProgrammerController.php
... lines 1 - 18
class ProgrammerController extends BaseController
{
... lines 21 - 76
public function listAction(Request $request)
{
$filter = $request->query->get('filter');
$qb = $this->getDoctrine()
->getRepository('AppBundle:Programmer')
->findAllQueryBuilder($filter);
... lines 84 - 89
}
... lines 91 - 189
}

In ProgrammerRepository, add a $filter argument, but make it optional:

41 lines src/AppBundle/Repository/ProgrammerRepository.php
... lines 1 - 8
class ProgrammerRepository extends EntityRepository
{
... lines 11 - 28
public function findAllQueryBuilder($filter = '')
{
... lines 31 - 38
}
}

Below, set the old return value to a new $qb variable. Then, if ($filter) has some value, add a where clause: andWhere('programmer.nickname LIKE :filter OR programmer.tagLine LIKE filter'). Then use setParameter('filter' , '%'.$filter.'%'). Finish things by returning $qb at the bottom:

41 lines src/AppBundle/Repository/ProgrammerRepository.php
... lines 1 - 8
class ProgrammerRepository extends EntityRepository
{
... lines 11 - 28
public function findAllQueryBuilder($filter = '')
{
$qb = $this->createQueryBuilder('programmer');
if ($filter) {
$qb->andWhere('programmer.nickname LIKE :filter OR programmer.tagLine LIKE :filter')
->setParameter('filter', '%'.$filter.'%');
}
return $qb;
}
}

If you were using something like Elastic Search, then you wouldn't be making this query through Doctrine: you'd be doing it through elastic search itself. But the idea is the same: prepare some search for Elastic, then use an Elastic Search adapter with Pagerfanta.

And that's all there is to it! Re-run the test:

./bin/phpunit -c app --filter filterGETProgrammersCollectionPaginated

Oooh a 500 error: let's see what we're doing wrong:

Parse error, unexpected '.' on ProgrammerRepository line 38.

Ah yes, it looks like I tripped over my keyboard. Delete that extra period and run this again:

./bin/phpunit -c app --filter filterGETProgrammersCollectionPaginated

Hmm, it's still failing: this time when it goes to page 2. To debug, let's see what happens if we comment out the filter logic and try again:

./bin/phpunit -c app --filter filterGETProgrammersCollectionPaginated

Now it fails on page 1: that extra willnotmatch programmer is returned and that makes index 5 Programmer4 instead of Programmer5. When we put the filter logic back, it has that exact same problem on page 2. Can you guess what's going on here? Yeas! We're losing our filter query parameter when we paginate through the results. womp womp.

Don't Lose the Filter Parameter!

In the test, the URL ends in ?page=2 with no filter on it. We need to maintain the filter query parameter through our pagination. Since we have everything centralized in, PaginationFactory that's going to be easy. Add $routeParams = array_merge() and merge $routeParams with all of the current query parameters, which is $request->query->all(). That should take care of it:

58 lines src/AppBundle/Pagination/PaginationFactory.php
... lines 1 - 10
class PaginationFactory
{
... lines 13 - 19
public function createCollection(QueryBuilder $qb, Request $request, $route, array $routeParams = array())
{
... lines 22 - 33
$paginatedCollection = new PaginatedCollection($programmers, $pagerfanta->getNbResults());
// make sure query parameters are included in pagination links
$routeParams = array_merge($routeParams, $request->query->all());
... lines 38 - 56
}
}

Run the tests one last time:

./bin/phpunit -c app --filter filterGETProgrammersCollectionPaginated

And we're green for filtering!

Leave a comment!

  • 2016-07-29 Thierno Diop

    Hi I wanted to let you know that adding the filter prameter in the created links like you did cause two problems :
    1 - if you have the prameter page in the link sent by the client you'll have two page prameter in the link you generate in your code
    2 - this add in the generated link the filter prameter with the syntax : %5Bfilter%5D=ilterValue
    The solution i found is to do it like this :

    $createLink = function ($pageumber) use ($routeName, $routeParams,$request) {

    return $this->router->generate($routeName, array_merge(

    $routeParams,

    $request->query->all(),

    array("page" => $pageumber)

    ));

    };
    And don't forhget to write the array("page" => $pageumber) after $request->query->all() otherwise the page prameter in the query will override the $pageNumber in the merged array!!
    PS : You do great work really great

  • 2016-07-04 weaverryan

    Everything is possible of course :). But, there's no simple way to do this that I can think of - you would probably build some custom functionality to do this yourself. So, it depends on how badly you need this!

  • 2016-07-04 bblue

    Would it be possible to somehow dynamically add the parameters exposed by the serializer as (the only) valid search fields?