Test runners for AngularJS - how to run the tests from eclipse IDE and CI server without too much complication?
angularjs, automated-tests, continuous-integration, integration-testing, unit-testing
Solution
Karma is for unit testing your JS, regardless of whether it is using Angular or not. The ins and outs of unit testing with Karma are covered very well here: http://www.yearofmoo.com/2013/01/full-spectrum-testing-with-angularjs-and-karma.html. Yes, Karma opens and closes browser windows as needed and specified in the configuration file. If you don't want any browser windows opened, you can use PhantomJS. You can run Karma from within most any IDE that is capable of running an external script, or run it via the command line.
Protractor is for end-to-end (or E2E) testing of your project as a whole. It will open a browser window and click through the pages as though it were a user, entering data where you tell it to and looking for the specified results. Protractor is a bit more complicated than just writing some Jasmine, but the results are worth it. Like Karma, you can run Protractor from within most any IDE that is capable of running external scripts or via the command line.
Yeoman is a process management system that incorporates dependency management via Bower, task automation via Grunt, and project management via Yo. It will run your tests in Karma and Protractor, minify your JS, CSS, and HTML, compile everything into appropriate files (internal JS, external libraries, and CSS) and provide you with a complete package that can be deployed. The beauty of Yeoman is that it is not specific to any one IDE. Everything it does can be done by scripting in your IDE or via the command line.
Now, having said all of this about Yeoman, you do still have to write the tests (it won't magically come up with them for you) and learn to integrate it into your development routine, but it is definitely the way to go for JS development. Eclipse is fine for JS development, but you'll get better performance and ease of use (IMHO) from WebStorm.
As for how these all fit into CI like Jenkins, I believe that both Karma and Protractor output test results in a format that Jenkins can read and display. With the scripting possibilities in Jenkins you can configure it to run the build process each time your source control repository (you are using some sort of source control, aren't you?) changes and show those results on the Jenkins page. My office has a very similar setup and we use it daily. I'm not the guy that has to do the Jenkins configuration, but I do work with Yeoman (and thus Karma and Protractor) via WebStorm on a regular basis and have had very good results.
Problem
I am trying to figure out a simple way to run tests on angularjs application. I am new to the testing world, so it's a little hard to understand all the options and the difference between them. My goal: to be able to run the tests simply from within my IDE - Eclipse. And to tests the code on google chrome browser. I found jasmine to be the obvious choice for writing js unit tests. The problem is choosing a runner both for the jasmine tests and for e2e tests. Trying to keep it simple, I've come up with the following idea for a setup: Write the unit tests in jasmine, and the e2e tests in phantomjs and syn.js. Then configure eclipse to run phantomjs as an external tool, so that the output will go to the console in eclipse. I also plan to have a CI job in Jenkins, and to my understanding Jenkins can also run phantom, so theoretically this solution will work the same for CI. Alternatively, there are test running tools like Karma and Protractor. On one hand, they seem to be recommended, but on the other hand they seem to me like overkill in some cases. They require a lot of different tools/services/processes to be running in order to work, and it seems like a pain maintain all that setup if it breaks. To my understanding: protractor runs on webdriverjs which runs on nodejs, and it requires a selenium server to be running in the background, and on top of all that the selenium opens real browser windows which seems a little pointless as opposed to headless browser testing. Then there is Karma, that I did not yet fully understand what it's supposed to do. From what I've read it monitors the files in my project and whenever a file is changed it runs the tests. I'm not sure how it runs the tests - is it also using selenium? And lastly, there are grunt and yeoman, which I did not understand at all what they do and how they interact or fit together with the other tools I've listed. I would appreciate if someone could clarify what these different tools do, and how they fit together. Also, how would they fit with Jenkins as a CI server? Also if you could comment on my "simpler setup" - does it make sense? Am I missing something?