Back to Blog
Jan 2nd, 2014

Evolving RequireJS, Bower and Grunt

weaverryan avatar Leannapelham avatar

Written by weaverryan, and Leannapelham

Edit
Evolving RequireJS, Bower and Grunt

Tip

Instead of Grunt, you might want to check out Gulp! It's at least as powerful and a lot easier. Checkout our series Gulp! Refreshment for Your Frontend Assets.

A few weeks ago, Leanna and I were one of the lucky 600+ that attended SymfonyCon in Warsaw - one of the best conferences we've been to! We hung out with some of our best tech friends, watched Leanna win tech Jeopardy, and had the pleasure to meet a lot of new friends!

I also gave a Christmas-themed talk on a really neat subject: "[Cool like a Frontend] Developer(http://www.slideshare.net/weaverryan/cool-like-frontend-developer-grunt-requirejs-bower-and-other-tools-29177248)", renamed to "Deck the Halls with Grunt, RequireJS & Bower". And because examples are best, an example project from the presentation lives on GitHub: knpuniversity/symfonycon-frontend

If you're curious about this stuff and couldn't be there for my talk, go read those slides. Then come back. We have a new piece to talk about.

An Evolving Best Practice

Like most new tech, what makes this stuff tricky is the lack of real projects and best practices when you're learning it. That was the point of my talk: to give you something real to build off of.

But I also knew that my solution wouldn't end up being the best. In fact, a tip from Gediminas (of DoctrineExtensions fame) and some others have already led me to one big change.

Keeping the assets Directory at the Root of your Project

In my talk, I propose having a web/assets/ directory where you put all of your JS, CSS/SASS, fonts, etc. When you run Grunt (which runs the RequireJS optimizer), this is copied to web/assets-built, and then some changes are made to it. In the end, the only change we need to make to our Symfony project is to point all of our assets to /web/assets-built instead of /web/assets when we're in the prod environment.

But a better solution may be to put the assets directory at the root of your project. This has a few advantages:

. Any source files (like SASS files) aren't exposed to the web;

. You no longer need to worry about changing between pointing to /assets

and /assets-built in your Symfony project.

The second point is very nice. By the end of my presentation, I defined two important Grunt tasks:

  1. grunt - operates on web/assets and does some basic things like SASS compilation';
  2. grunt production - copies web/assets to web/assets-built and then does several things to it.

With this new setup, we would change this slightly:

  1. grunt - copies assets/ to web/assets and does some basic things like SASS compilation';
  2. grunt production - copies assets to web/assets and then does several things to it.

The difference is that - whether we're developing or deploying - our assets always live in web/assets. This means that you don't need any logic in your Symfony application to change paths from /assets/ to /assets-built. Developing? Just use grunt (or, more usefully grunt watch). Want to use the assets as they'll be built for production? Just run grunt production.

Changes to Gruntfile.js

If you want to try this, let's talk about the exact changes we need. It's a simple 3-step process. If you want to skip and see the end result, check out the assets-in-root branch on GitHub.

1) First, the easy part: move web/assets to assets. Awesome.

2) Next, update your Twig templates to simply point at the assets directory, replacing the "smarter" variable used before.

Before:

<script src="{{ asset(assetsPath~'/vendor/requirejs/require.js') }}"></script>
<script>
    requirejs.config({
        baseUrl: '{{ asset(assetsPath~'/js') }}'
    });
    // ...
</script>

After:

<script src="{{ asset('assets/vendor/requirejs/require.js') }}"></script>
<script>
    requirejs.config({
        baseUrl: '{{ asset('assets/js') }}'
    });
    // ...
</script>

3) Modify the Gruntfile.js to copy assets/ to web/assets and then operate entirely on the web/assets directory.

Ok, this part isn't so simple. First, you'll need a new Grunt plugin: grunt-contrib-copy by adding it to package.json, and importing its tasks in Gruntfile.js:

// Gruntfile.js
// ...
module.exports = function (grunt) {
    // ...

    grunt.loadNpmTasks('grunt-contrib-copy');
    // ...
};

With some configuration, this will copy one directory (e.g. assets) to another directory (web/assets). We've been relying on RequireJS to do this until now, but I now want something that will copy these files, even if I'm not using the RequireJS optimizer:

// Gruntfile.js
// ...

copy: {
    main: {
        files: [
            {
                expand: true,
                src: ['assets/**'], dest: 'web'}
        ]
    }
},
// ...

With this, we now have a new grunt copy command, which will copy assets/ to web/assets. That's not very useful on its own, but we can now point all the other tasks in Gruntfile.js to operate on the web/assets directory, including Compass, JSHint and RequireJS.

We also have two "watch" sub-commands that guarantee that JSHint is run whenever JavaScript files change and Compass whenever .scss files change. We'll continue to have the watch sub-task look for file changes in the assets/ directory at the root of our project, since that's where we edit files. But before running jshint or compass, each will call copy first, to copy things into web/assets:

// Gruntfile.js
// ...

watch: {
    scripts: {
        files: ['assets/js/**'],
        tasks: ['copy', 'jshint']
    },
    // watch all .scss files and run compass
    compass: {
        files: 'assets/sass/*.scss',
        tasks: ['copy', 'compass:dev'],
        options: {
            spawn: false
        }
    }
}

The setup probably still has a few imperfections, but to see it all put together, see the grunt-contrib-copy branch on GitHub. This setup adds a small amount of complexity, since you must copy files every time any change is made, even while developing. But since this is all handled in Grunt and grunt watch, we only feel that complexity when we're first getting things configured.

Cleaning up SASS and old Files

I've also been talking with a Matt Davis, we brought up some more potential improvements/problems:

  1. The SASS files no longer live in web/, but are still copied to web/ when Grunt runs. If you really want to hide these files, you'll need to omit them from the copy task, or remove them afterwards.

  2. If you delete a file from assets/, it will still live in web/assets/, because the copy task copies new files, but nothing ever removes the old files.

The answer to both of these is the grunt-contrib-clean plugin.

Tip

The solution to this has been even further evolved to never copy the sass files at all. Just check out the assets-in-root branch on GitHub or pull request #7 for more details. Thanks to Daniel Paschke for the tips.

First, install it like any Grunt plugins:

$ npm install grunt-contrib-clean --save-dev

Then activate its tasks in Gruntfile.js:

// Gruntfile.js
module.exports = function (grunt) {
    // ...
    grunt.loadNpmTasks('grunt-contrib-clean');
    // ...
};

We'll create 2 subtasks: one for cleaning out web/assets before copying and another for cleaning out the web/assets/sass directory after copying:

// Gruntfile.js
// ...

grunt.initConfig({
    clean: {
        build: {
            src: ['<%= targetDir %>/**']
        },
        sass: {
            src: ['<%= targetDir %>/sass']
        }
    },
});

// ...
// sub-task that copies assets to web/assets, and also cleans some things
grunt.registerTask('copy:assets', ['clean:build', 'copy', 'clean:sass']);

// the "default" task (e.g. simply "Grunt") runs tasks for development
grunt.registerTask('default', ['copy:assets', 'jshint', 'compass:dev']);

// register a "production" task that sets everything up before deployment
grunt.registerTask('production', ['copy:assets', 'jshint', 'requirejs', 'uglify', 'compass:dist']);

We've also created a new convenience task: copy:assets, which cleans web/assets, copies assets/ to web/assets/, then removes web/assets/sass. Phew! Just make sure that this new copy:assets is the first step in our default and production tasks. Now, when we run grunt or grunt production, all the copying and cleaning will happen first.

Other Improvements?

This was the first big change that I've come across, but if you see other improvements, I'd love to hear them!

Have fun!

19 Comments

Sort By
Login or Register to join the conversation

Looks great - you rock!

| Reply |
Default user avatar Pablo Molina Toledo 5 years ago

Hi, I'm trying to configure this work environment but using LESS instead of Sass. I would like to compile and watch my source files following the same folder structure. Do you know how to do a Grunt task to do this?

Thanks.

| Reply |

Hi Pablo!

I think it should be a lot of the same basic things. First, you'll probably use https://github.com/gruntjs/... instead of the "compass" module that I'm using. Your first goal would just be to be able to say "grunt less" and have it compile your less for you. Then, the only extra step would be to get "grunt watch" working. That should look similar to my watch: https://github.com/knpunive... - except that you'll look at a less directory and call your "less" (or perhaps less:dev) task instead of "compass:dev".

I'm sure there a lot of other little details, but hopefully this gets you rolling :). I've never done it personally, so it's all I can say. Let me know how it goes!

Cheers!

1 | Reply |
Default user avatar Pablo Molina Toledo weaverryan 5 years ago

Hi again, thank you very much for your answer. I was on vacation ;) so now is that a read the email.
I'll try your solution but with a few modifications because I want to store the bundle's assets in the original folder.

Thanks anyway!!!

| Reply |
Default user avatar Marc Höffl 5 years ago

Hi,

I am developing a FullStackFrontend App with Ember JS and my S2 Application is purely rest API. So not really using Twig for showing my Application.

For building my frontend I use grunt, which also creates new file versions etc. (Its standard yeoman generated gruntfile). The problem is now, that I need to deploy the whole thing to my cloud provider. I currently have both applications (Ember in the Frontend, S2 in the backend) totally seperated, but I need now to get them together somehow in order to deploy them on the same server. I thought that it might make sense to put all frontend stuff into my backend application and then create a .htaccess file that serves always my frontend except if the url is starting with /api/.

The only problem is how do I get my frontend grunt build index.hmlt from my frontend "dist" file into the main directoy of my web folder? Since I am not only talking about JS / CSS etc. assets that could also live in another folder or somewhere. I could somehow put my whole frontend application as git submodule into my backend repo. But I am not really sure where and then how to really get them together. Any ideas on that?

| Reply |

Hi Marc!

First, I was thinking the same thing as you: an .htaccess rule that points to index.html for everything except for /api URLs (which would point to app.php). I would definitely go with this solution :).

It sounds like the other problem is just a matter of: how best to "mix" these 2, standalone applications for the purpose of deployment, though perhaps I'm over-simplifying things. If you want to keep the 2 apps separate (which I think makes nice sense), then I would write a little deployment script that "builds" the 2 together up on the server. A really simple example of what the little build script would do is:

1) Clone (or pull) the Symfony application into a directory on the server
2) Clone (or pull) the Ember JS application into a different directory on the server
3) Run the grunt build items you need
4) Copy your build Ember app (index.html and all the subdirectories of assets) into the web directory over the Symfony application.

As you already know, the steps, technically speaking, are really simple. So I think, why not just write a simple script that just does the steps you need. Of course, you can get more sophisticated if you like: write the script using Ant build tasks, or run this whole process on a different server (so that you don't need to run Grunt on production) and then rsync the finished product to your server, etc etc. So I don't have any magic answer, other than write a script that puts the 2 pieces together :).

I hope that helps at least a little - let me know! Cheers!

| Reply |
Default user avatar Jonatas Freitas 5 years ago

Hello,

to make independent bundles, i put the js files that will be used by views within Resources/public/js , how to use it in this structure ?

Thx a lot.

| Reply |

Hey Jonatas!

First, check out http://knpuniversity.com/sc... - it's the updated version of this (using Gulp instead of Grunt, which I prefer these days). To answer your question, if you aren't creating independent bundles (i.e. if the bundles are meant to all be used in one application and not shared), then I wouldn't put them in Resources/public/js of the bundle - there's no advantage to doing this (since the bundles are coupled to your app anyways). But, if you *are* creating independent bundles, then you really need to treat these bundles like treat, standalone libraries. This means that each bundle would have its own gulp/grunt setup, with the end goal of creating one, minified CSS/JS file for the files in just that *one* bundle. Then, when you integrate these bundles into your project, the project will have its own grunt/gulp setup that would combine/process all of those files.

I hope that helps!

| Reply |
Default user avatar Heiko Krebs 5 years ago

First of all: thanks for your post! So all your "frontend-files" like css/sass/js/images live in your assets directory and no longer within a bundle-directory (Resources/public)? Isnt this a problem regarding the concept of bundles?

| Reply |

Hey Heiko Krebs!

Yes, great question. You could of course put things inside of a bundle, instead of inside the "assets" directory (which lives outside the bundle). The choice is yours, and technically speaking, there's no difference of course.

The reason you'd choose one over the other is purely philosophical. Is this CSS or JS file really specific to one bundle? Or I am building one single global frontend that really has nothing to do with the bundle organization? If you think about a frontend developer, they don't necessarily care or understand your 5 bundles organization - they just want to build the 1 frontend app and organize things how it makes sense for them. So from that standpoint, I like to remind people that this is a perfectly valid option. Of course, if you really do have bundles that have their own JS/CSS, or if those bundles are going to be shared, then definitely put things in the bundle.

Cheers!

1 | Reply |

so how would a setup would look like, if we have bundle specific javascript for example.

like if we stay on the yoda events example lets say we have some user-bundle specific javascript for f-e validation of the registration form. (thats something thats only needed in the userbundle).

how would you change the way this works?

can we tell the grunt task to do an assetic-dump for us, or even use its contrib-copy task to perform a copy the the /assetics directory before we do the other grunt tasks?

| Reply |

Hey paschdan!

Hmm. So, you *could* not use Grunt for those assets you have in a bundle. For example, you could include those directly in your template, or use Assetic for just those few assets. What I mean is, there's no reason to need to push everything through Grunt (though if everything fits nicely through Grunt, that's awesome). Even RequireJS should work fine - you can just add a "paths" entry that points to the bundle assets (the path would look something like ../../bundles/user/...) and it will load it just fine.

Thinking about it, Grunt gets complicated when you want to Uglify or SASS process the assets (since this modifies the originals). In those cases, it seems like you may need to use grunt-copy or grunt-sync to move them first.

So, I can't see all of the little challenges that you'd see when having some assets in bundles. But, I'd start simple - consider not having those in Grunt, and then little by little, work them into Grunt as I have my needs. And of course, with grunt-copy or grunt-sync, there's nothing that you can't move into an "assets" directory so that it works like anything else.

I hope that helps - good luck with it!

| Reply |

Hi Ryan , great stuff! Just a quick question. I have been trying to get FOSJsRoutingBundle working with the template: In my _requirejs.html.twig I have the following:

http://pastebin.com/tj6ZVm9Y

And in my common.js the following:

http://pastebin.com/0YqDNxcF

Finally in a module I defined I have:

http://pastebin.com/ddazJW1N

However I am getting the following error:

Uncaught TypeError: undefined is not a function.

It looks like it Routing was not loaded correctly :(

Any help would be greatly appreciated!

| Reply |

Hey Daniel!

The basic ideas of the setup look fine to me. The only possible problem I see is in your test module - 'routing' if y our 5th entry in define(), but it's the second argument to your function - function($, Routing). I'm pretty sure that Routing is actually being set to "domReady" - not routing (since it is the second entry in your define array).

I hope that helps! Cheers!

| Reply |

Hi Ryan, yes that was it thanks for your help!

| Reply |
Default user avatar paschdan 5 years ago edited

I got another small improvement, when working with assets_base_urls you should use this code in _requirejs.html.twig:


//...
baseUrl: '{{ asset('/assets/js') }}'
//...

when using assets in root

| Reply |

Hey paschdan!

You're right! But let me improve that just a little bit further :)


baseUrl: '{{ app.request.basePath }}/assets/js'

We're both fixing the issue with having your document root not at the root of your host (e.g. http://localhost/gruntplay/web/app_dev.php). But the problem with asset() is that if you configure a CDN in config.yml, you will end up with a baseUrl that looks like this: 'http://pathtosomecdn/assets/js&#039;, which I don't believe will work out :).

I've just pushed this fix to the code and updated this post (https://github.com/knpuniversity/blog/commit/3f1f6fb523eb75a090e89037e82ba810c56967b3).

Thanks!

| Reply |

Hi Peter Siska!

I see! So in this case, since you can't have prod assets and dev assets in the same directory, you're going to have to have some sort of an environment switch in your templates, kind of like we do temporarily here: http://www.slideshare.net/w...

The problem is that you need to remember to do this, which is especially annoying if you are including lots of CSS files everywhere or image files (though easy for JS, if you're using RequireJS - it's just this one spot). One solution might be to use the `asset()` function in a clever way. For example, there is already an "asset_version_format" config key that allows you to rewrite URLs, for example:

{{ asset('css/foo.css') }}

will be output as /assets/css/foo.css with the right asset_version_format configuration. Check out: http://symfony.com/doc/curr... - especially the gray sidebar below that talks about using some clever things to modify the path in this way.

I haven't tried this, but it should work - I *do* use a trick like this (where I modify the actual path using asset_version_format) for cache busting.

Let me know if that helps!

| Reply |

Hey Peter!

Awesome, thanks for following up with your solution - I'm sure it'll help people! About the `files tweak you suggested, do you have a working copy of the solution? I originally *was* trying the expanded configuration, but iirc, I had issues because of how uglify works (I can't remember the details now). So if you *do* have it working, then you've done something I couldn't - and I would love to see your solution and integrate it! You could even open a PR against https://github.com/knpunive... ;)

Cheers!

| Reply |

Delete comment?

Share this comment

astronaut with balloons in space

"Houston: no signs of life"
Start the conversation!