Buy

Using a Test Database

We're using the built-in PHP web server running on port 8000. We have that hardcoded at the top of ApiTestCase: when the Client is created, it always goes to localhost:8000. Bummer! All of our fellow code battlers will need to have the exact same setup.

We need to make this configurable - create a new variable $baseUrl and set it to an environment variable called TEST_BASE_URL - I'm making that name up. Use this for the base_url option:

273 lines src/AppBundle/Test/ApiTestCase.php
... lines 1 - 45
public static function setUpBeforeClass()
{
$baseUrl = getenv('TEST_BASE_URL');
self::$staticClient = new Client([
'base_url' => $baseUrl,
'defaults' => [
'exceptions' => false
]
]);
... lines 55 - 59
}
... lines 61 - 273

There are endless ways to set environment variables. But we want to at least give this a default value. Open up app/phpunit.xml.dist. Get rid of those comments - we want a php element with an env node inside. I'll paste that in:

36 lines app/phpunit.xml.dist
... lines 1 - 3
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/4.1/phpunit.xsd"
backupGlobals="false"
colors="true"
bootstrap="bootstrap.php.cache"
>
... lines 10 - 17
<php>
<env name="TEST_BASE_URL" value="http://localhost:8000" />
</php>
... lines 21 - 34
</phpunit>

If you have our setup, everything just works. If not, you can set this environment variable or create a phpunit.xml file to override everything.

Let's double-check that this all works:

phpunit -c app --filter testGETProgrammersCollection src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php

Tests Killed our Database

One little bummer is that the tests are using our development database. Since those create a weaverryan user with password foo, that still works. But the cute programmer we created earlier is gone - they've been wiped out, sent to /dev/null... hate to see that.

Configuring the test Environment

Symfony has a test environment for just this reason. So let's use it! Start by copying app_dev.php to app_test.php, then change the environment key from dev to test. To know if this all works, put a temporary die statement right on top:

31 lines web/app_test.php
<?php
die('working?');
... lines 3 - 24
$kernel = new AppKernel('test', true);
... lines 26 - 31

We'll setup our tests to hit this file instead of app_dev.php, which is being used now because Symfony's server:run command sets up the web server with that as the default.

Once we do that, we can setup the test environment to use a different database name. Open config.yml and copy the doctrine configuration. Paste it into config_test.yml to override the original. All we really want to change is dbname. I like to just take the real database name and suffix it with _test:

21 lines app/config/config_test.yml
... lines 1 - 17
doctrine:
dbal:
dbname: "%database_name%_test"

Ok, last step. In phpunit.xml.dist, add a /app_test.php to the end of the URL. In theory, all our API requests will now hit this front controller.

Run the test! This shouldn't pass - it should hit that die statement on every endpoint:

phpunit -c app --filter testGETProgrammersCollection src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php

They fail! But not for the reason we wanted:

Unknown database `symfony_rest_recording_test`

Woops, I forgot to create the new test database. Fix this with doctrine:database:create in the test environment and doctrine:schema:create:

php app/console doctrine:database:create --env=test
php app/console doctrine:schema:create --env=test

Try it again:

phpunit -c app --filter testGETProgrammersCollection src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php

Huh, it passed. Not expected. We should be hitting this die statement. Something weird is going on.

Debugging Weird/Failing Requests

Go into ProgrammerControllerTest to debug this. We should be going to a URL with app_test.php at the front, but it seems like that's not happening. Use $this->printLastRequestUrl() after making the request:

73 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 53
public function testGETProgrammersCollection()
{
... lines 56 - 64
$response = $this->client->get('/api/programmers');
$this->printLastRequestUrl();
... lines 67 - 70
}
... lines 72 - 73

This is one of the helper functions I wrote - it shows the true URL that Guzzle is using.

Now run the test:

phpunit -c app --filter testGETProgrammersCollection src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php

Huh, so there's not app_test.php in the URL. Ok, so here's the deal. With Guzzle, if you have this opening slash in the URL, it takes that string and puts it right after the domain part of your base_url. Anything after that gets run over. We could fix this by taking out the opening slash everywhere - like api/programmers - but I just don't like that: it looks weird.

Properly Prefixing all URIs

Instead, get rid of the app_test.php part in phpunit.xml.dist:

36 lines app/phpunit.xml.dist
... lines 1 - 3
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
... lines 5 - 17
<php>
<env name="TEST_BASE_URL" value="http://localhost:8000" />
</php>
... lines 21 - 34
</phpunit>

We'll solve this a different way. When the Client is created in ApiTestCase, we have the chance to attach listeners to it. Basically, we can hook into different points, like right before a request is sent or right after. Actually, I'm already doing that to keep track of the Client's history for some debugging stuff.

I'll paste some code, and add a use statement for this BeforeEvent class:

283 lines src/AppBundle/Test/ApiTestCase.php
... lines 1 - 10
use GuzzleHttp\Event\BeforeEvent;
... lines 12 - 20
class ApiTestCase extends KernelTestCase
{
... lines 23 - 46
public static function setUpBeforeClass()
{
... lines 49 - 59
// guaranteeing that /app_test.php is prefixed to all URLs
self::$staticClient->getEmitter()
->on('before', function(BeforeEvent $event) {
$path = $event->getRequest()->getPath();
if (strpos($path, '/api') === 0) {
$event->getRequest()->setPath('/app_test.php'.$path);
}
});
... lines 68 - 69
}
... lines 71 - 281
}

Ah Guzzle - you're so easy to understand sometimes! So as you can probably guess, this function is called before every request is made. All we do is look to see if the path starts with /api. If it does, prefix that with /app_test.php. This will make every request use that front controller, without ever needing to think about that in the tests.

Give it another shot:

phpunit -c app --filter testGETProgrammersCollection src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php

Errors! Yes - it doesn't see a programmers property in the response because all we have is this crumby die statement text. Now that we know things hit app_test.php, go take that die statement out of it. And remove the printLastRequestUrl(). Run the entire test suite:

phpunit -c app

Almost! There's 1 failure! Inside testPOST - we're asserting that the Location header is this string, but now it has the app_test.php part in it. That's a false failure - our code is really working. Let's soften that test a bit. How about replacing assertEquals() with assertStringEndsWith(). Now let's see some passing:

phpunit -c app

Yay!

Leave a comment!

  • 2016-10-11 weaverryan

    No worries - thanks again for the post!

  • 2016-10-10 Bruno Lima

    Thank you. And sorry. :) It looks like I posted my comment on the wrong place. I'm already on course 4, but I came back here in order to remember that app_test.php prefix and to find out if this good piece of code was being underused or if I did something wrong.

  • 2016-10-10 weaverryan

    Hey Bruno!

    Nice looking into this - I think you're right! At one point, I think we're *not* yet using the test environment, and I think I setup this nice little feature during that :). So, this comes down to 2 things (and thanks for pointing it out):

    1) It looks like the code for Symfony3 / Guzzle6 should be updated to avoid that Array to string conversion. I can check into that.

    2) I agree to *not* enable the profiler for tests - it does really slow things down. But, you could - if you were having some issues - enable it temporarily, which is when this line would kick into shape and help give you the URL to the profiler. I should add a note about this - it's a nice feature, but it needs to be highlighted.

    So, I need to look into the code still, but it seems like you're absolutely right. Thanks for asking about this!

    Cheers!

  • 2016-10-09 Bruno Lima

    Hi Ryan, I have a question: Since the default `config_test.yml` disables the Symfony web profiler, this `if` clause will never true on `ApiTestCase.php` (the most recent version, so far, already on Symfony 3):

    ```
    $profilerUrl = $response->getHeader('X-Debug-Token-Link'); // header not provided
    if ($profilerUrl) { // never evaluated as true
    $fullProfilerUrl = $response->getHeader('Host').$profilerUrl[0]; // Results in "Array to string conversion"
    ```

    So one can't realize that `$response->getHeader('Host')` returns an array and can't be converted to string. Only `$profilerUrl[0]` is fine.
    Did you changed the configuration or stopped using the web profiler? Is it worth to enabled it? Symfony itself recommends disable for performance.

    Thank you.

  • 2016-09-08 weaverryan

    Hey Tael!

    Ah, you're of course right :). I made this change recently, but I totally reversed it! Thanks for catching it and commenting. I've just made the fix (https://github.com/knpuniversi... and am deploying it now!

    Cheers!

  • 2016-09-08 Tael Kim

    If I doesn't wrong,
    this downloaded code will be fixed.

    /src/AppBundle/Test/ApiTestCase.php
    ----------------------------
    62 $baseUrl = getenv('TEST_BASE_URL');
    63 if ($baseUrl) {
    64 static::fail('No TEST_BASE_URL environmental variable set in phpunit.xml.');
    65 }
    ----------------------------
    TO
    ----------------------------
    62 $baseUrl = getenv('TEST_BASE_URL');
    63 if (!$baseUrl) {
    64 static::fail('No TEST_BASE_URL environmental variable set in phpunit.xml.');
    65 }
    ----------------------------

    if baseUrl loaded correctly, It's no error. doesn't it?

    please check and reply weaverryan :D

  • 2016-06-29 weaverryan

    Great news :) - thanks for the update!

  • 2016-06-29 Stas Goshtein

    Hi, Ryan and Victor. Just wanted to give you an update.
    I started everything from scratch and now it is working perfectly. I suppose it was some extra char in one of Yamls.
    Or may be the fact, that I also copied routes_dev.yml to routes_test.yml

  • 2016-06-22 Victor Bocharsky

    Ah, if your DEV is working perfectly, most likely my advise doesn't help. (

    BTW, when you upgrade Guzzle up to 6, did you change "base_url" to the "base_uri" for Guzzle client?

  • 2016-06-22 Stas Goshtein

    Hi, Victor, thank you for your answer.
    Yes, my DEV is working perfectly, all tests are now running on it without problem.
    I will try what you suggested about NGINx config and will keep thread updated.

  • 2016-06-22 Victor Bocharsky

    Hey Stas!

    I suppose your PROD environment works well, right? Does your app work in DEV environment correctly? I mean can you serf /app_dev.php/api/core/actions/add directly? It's strange if it works in DEV but not in TEST.

    If app does not work in DEV too, then could you try to use for ^/(app_dev|app_test|config)\.php(/|$) the same configuration directives which are used in ^/app\.php(/|$)? And comment out every internal; directive in your Nginx Symfony config for debug purpose. And don't forget to restart Nginx after any config changes!

    Cheers!

  • 2016-06-22 Stas Goshtein

    Hi, Ryan, thanks for replying first of all. Getting the same message from browser and also from Postman. I tried about 5 different NGINX configs as well. Cleared caches few time - no success for now. Will probably run Unit testing from Jenkins on another server to keep my DEV database.

  • 2016-06-21 weaverryan

    Hi Stas!

    Ah, interesting... So the key thing I'm looking at is the error from Symfony: No route found for GET /app_test.php/api/core/actions/add. That app_test.php part should *not* be in there. Obviously, we *do* go to /app_test.php/api/core/actions/add, but since app_test.php is a PHP file that's executed, Symfony ultimately thinks that the URL is /api/core/actions/add. So having the "app_test.php" in the error message is our clue. I can think of 2 ways this is happening:

    1) Somehow, we're actually requesting /app_test.php/app_test.php/api/core/actions/add
    2) Something is misconfigured in Nginx, and so even though app_test.php is being executed, some bad information is being placed into $_SERVER.

    I assume this all works if you just surf to the URL (/app_test.php/api/core/actions/add) directly? If so, we need to really be sure that the URL that's being requested is correct. I know the top of the error messages says you made a request to /api/core/actions/add... but I'm not sure I trust it :)

    Cheers!

  • 2016-06-19 Stas Goshtein

    Hi, guys, having some stupid problem, after following your chapter and also applying Ryan's new APITestCase for Guzzle 6 and Symfony3, my app won't find routes in test env.

    Failure! when making the following request:
    GET: http://api.lovelock.ws/api/cor...

    Server: nginx/1.10.1
    Content-Type: application/problem+json
    Transfer-Encoding: chunked
    Connection: keep-alive
    X-Powered-By: PHP/7.0.7
    Cache-Control: no-cache
    Date: Sun, 19 Jun 2016 12:48:04 GMT
    {
    "detail": "No route found for \"GET \/app_test.php\/api\/core\/actions\/add\"",
    "status": 404,
    "type": "about:blank",
    "title": "Not Found"
    }
    F 1 / 1 (100%)

    Time: 3.32 seconds, Memory: 18.00MB

    There was 1 failure:

    1) APIBundle\Tests\Controller\CoreActionControllerTest::testCoreActionGetByName
    Failed asserting that 404 matches expected 200.

    Have to mention, that test DB is working and test.log is created and written.
    My server config is NGINX/FPM and config is default recommended by NGINX, however I only change this line to pass app_test.php to FPM:

    location ~ ^/(app_dev|app_test|config)\.php(/|$) {
    .....
    }

  • 2016-06-09 weaverryan

    You're absolutely correct about the absolute paths (and I *do* think this was something that was added in some more recent versions of Guzzle). To get around it, we use a middleware that adds the app_test.php even when the URL starts with a slash. It's an annoying little thing to need to add, but it works pretty well. Here's the Guzzle 6 version for those who are curious: https://gist.github.com/weaver...

    Cheers!

  • 2016-06-09 dermeck

    Hey I just had a similar problem although the response was not empty Guzzle was kind of ignoring the "/app_test.php" part of the base url. The problem was that in my setup (Symfony 3, Guzzle ^6.2) Guzzle overrides the path if the request uses an absolute path like "/api/...". Using "api/..." instead solved the problem for me.

    Since it worked in the video I suppose this might be an issue with the later versions.

  • 2016-04-27 weaverryan

    Hi Roy!

    Hmm, a few things to check:

    1) Try renaming (in phpunit.xml.dist) the variable to something different - e.g. TEST_BASE_API_URL. I just want to make sure nothing is overriding the other value :).

    2) Make sure your phpunit.xml.dist looks exactly like mine - you can see the entire file if you expand the code blocks on this page.

    Let me know if you find anything out!

  • 2016-04-26 Roy Hochstenbach

    Setting the 'TEST_BASE_URL' value in phpunit.xml.dist doesn't seem to be working. getenv('TEST_BASE_URL') returns an empty response.

  • 2016-02-22 weaverryan

    Hi Mihail!

    Hmm, 2.62 seconds for 3 functional tests (each includes a real HTTP request) seems *really* good to me. If all your tests run that fast, you could have hundreds of tests and they would still execute in just a few minutes. Also, if there are performance gains you can get, it's possible that the database is not the problem. If you're interested, I'd recommend using Blackfire.io from the command line. I've never tried it before (I've wanted to), but it should be able to profile your tests being run: telling you exactly what is and is not taking up performance.

    Cheers!

  • 2016-02-18 Mihail

    Hi Ryan!

    I try to follow this tutorial by using SF2.7 and modified ApiTestCase to extend WebTestCase using the idea from Symfony Jobeet tutorial for SQLite. https://gist.github.com/bamper...

    Now all 3 tests work with test.sqlite database at this point, but a little bit slower: Time: 2.62 seconds, Memory: 27.75Mb

    Would you be so kind to look at this file and give some advice on what can be improved further and whether there are any drawbacks with this approach?

  • 2016-02-09 weaverryan

    Hey again!

    No worries :). Sorry for the late reply - this was a tougher question initially and I had to get back from a conference!

    To answer your question directly: I don't know (but I could probably find out). But, I might have a simpler way. Have you tried:

    A) Creating the file sqlite database.
    B) Copy it to something like myDb.template
    C) At the beginning of each test, don't rebuild the DB, just copy the file from myDb.template to myDb (assuming that's your real DB name in the test environment). Do this *before* you boot your kernel.
    D) Run your tests, which will use the already-perfectly-populated new database file.

    I don't know if this works (I can't think why no?) - but I've often wondered if it would! And if so, I think it would be faster than an in-memory database that you need to rebuild on every test.

    Cheers!

  • 2016-02-02 zacball

    Wow, thanks for the full length reply. I should have been more specific in my question. I actually wasnt following along with the entire series, was just looking to cherry pick some info I was looking for. :) I am using the client object so an example of one of my tests looks like this: https://gist.github.com/zball/...

    I wanted to avoid using a separate testing db, and just wanted to use an in memory version of sqlite. Currently the sqlite is stored in a file and the db is created and destroyed on every test to ensure that every single test is run on a nice clean version of the db. It works, but its SUPER slow.

    This is config_test.yml: https://gist.github.com/zball/.... If I change the path to :memory:, nothing happens.

    Anyways, I greatly appreciate your previous reply and if what Im attempting just isnt possible, doesnt make any sense, or is out of the scope of the video just let me know.

  • 2016-02-02 weaverryan

    Hi there!

    Ok, great question. First, using the "memory" SQLite database will *not* be possible if you're testing how we are - i.e. using something like Guzzle. Here's why:

    1) In the test file, PHP initializes, you boot Symfony and tell it to create an in-memory database with some code in it.
    2) Still in the file, you use Guzzle to make an *external* HTTP request to Symfony. The key is "external" - this is literally the same as making a web request to some code that lives on an entirely different server. Sure, your code all lives together on the same machine - but since this is an external, real HTTP request, your web server gets called and *it* loads up a new *fresh* PHP process and boots a *new* Symfony application. That HTTP request that's processing and *your* test code are not in the same PHP process: they're totally independent and cannot share anything in memory.

    So, it's not Doctrine's issue or anything else - it's just not possible. Now, if you use Symfony's built-in testing tools - i.e. its Client object that's covered in the Symfony docs, then you *can* do this. That's because those are "fake" HTTP requests: it uses your already-booted Symfony app and sends requests into it. In this case there is just *one* PHP process and you *can* share memory. However, I prefer using real HTTP requests via Guzzle... because for me, this is simpler and you're really testing your API like a real, external user.

    To accomplish what you want, I would try something like this:

    A) Use a file-based SQL file
    B) Load up that file with all the tables+data you want (i.e. you could run your fixtures to populate this file).
    C) Copy this file to some other filename - e.g. db.template
    D) At the *very* beginning of your test (probably even before Symfony is booted), copy "db.template" to whatever your *real* database file name should be (e.g. main.db). Then, at the end of the test, simple delete your "main.db" file to reset things.

    I would still recommend sending your web requests through app_test.php, because you can then configure the test environment to use a different database filename (e.g. main_test.db)... which is nice only so that when you test, it doesn't completely kill your development database.

    I think this is an issue we need to make *very* easy in Symfony - it's a really nice setup.

    Cheers!

  • 2016-02-02 zacball

    I have been trying to accomplish testing using sqlite and having no luck. Here is an example from others of my issue: https://github.com/doctrine/db... . Is there a way to accomplish this?