docker-compose up can't find module, but running from bash works
docker, docker-compose, node.js
Solution
The problem is that the `node_modules` volume in compose file is set as anonymous, anonymous volumes are preserved when containers are recreated, even with `--build`, the node still can't find the newly added module. See issues here.
For now, I use these two commands to make it work since the docker-compose v1.9 with --renew-anon-volumes witch I think should be used to handle this situation is not officially released yet:
docker-compose down
docker-compose up --build
Problem
I've been building a Node.js app on Docker, and following the advice in John Lees-Miller's excellent Lessons from Building a Node App in Docker for setting up my environment. Everything has been working fine for a week or two, but today I started running into an issue that I cannot figure out. In a nutshell, I can no longer run the app via `docker-compose up`; I'm getting a node error for newly added module depedencies: ``` sfsftp_1 | module.js:327 sfsftp_1 | throw err; sfsftp_1 | ^ sfsftp_1 | sfsftp_1 | Error: Cannot find module 'eval' sfsftp_1 | at Function.Module._resolveFilename (module.js:325:15) sfsftp_1 | at Function.Module._load (module.js:276:25) sfsftp_1 | at Module.require (module.js:353:17) sfsftp_1 | at require (internal/module.js:12:17) sfsftp_1 | at Object.<anonymous> (/home/app/sfsftp/lib/reformatter.js:4:13) sfsftp_1 | at Module._compile (module.js:409:26) sfsftp_1 | at Object.Module._extensions..js (module.js:416:10) sfsftp_1 | at Module.load (module.js:343:32) sfsftp_1 | at Function.Module._load (module.js:300:12) sfsftp_1 | at Module.require (module.js:353:17) ``` However, if I run a bash shell: ``` docker-compose run --rm sfsftp /bin/bash ``` and start the node app from inside the container with `node sftp.js`, the app starts up fine! I initially had this issue earlier today with the module `jsonminify`, which I had recently added to the project; I believe I had run the app both ways (`docker-compose up` and `docker-compose run` via bash) since making that change, but perhaps not? In the interest of expediency, I decided to punt on the issue, and removed my use on `jsonminify`. Later, I needed to add the `eval` module to my project, and now I'm getting the same error for that module. I feel like the `docker-compose up` command is using an older copy of the image than the `docker-compose run` command, one without my recent changes. When I've added the modules, I've followed the process in the article - use `docker-compose run` to get a bash shell and running `npm install --save module` inside the running instance. My `package.json` and `npm-shrinkwrap.json` files are being updated, and I see the `eval` module in both. I'm still fairly new to developing in Docker containers, so I've reached the point of "let's try this"; so far I've tried: - `docker-compose build` followed by `docker-compose up` - `docker-compose up --build` - `docker-compose build --no-cache` followed by `docker-compose up` - `docker run -p 22:9001 sftpdocker_sfsftp` The last one worked - running docker directly, instead of via docker-compose. But this loses some of the benefits of the setup. How do I make docker-compose work? For reference, here is my `Dockerfile`: ``` # prepared with reference to http://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html # Current LTS Node version FROM node:4.6.0 # Let's not run as Root. And, update to recent NPM RUN useradd --user-group --create-home --shell /bin/false app &&\ npm install --global npm@3.7.5 ENV HOME=/home/app # Get what we need for npm install COPY package.json npm-shrinkwrap.json $HOME/sfsftp/ RUN chown -R app:app $HOME/* # and install dependencies USER app WORKDIR $HOME/sfsftp RUN npm install && npm cache clean # do this after dependencies, so that if only the app changes, npm install won't be rerun USER root COPY . $HOME/sfsftp RUN chown -R app:app $HOME/* USER app # Start it up! CMD ["node", "sftp.js"] ``` and `docker-compose.yml`: ``` sfsftp: build: . ports: - '22:9001' volumes: - .:/home/app/sfsftp - /home/app/sfsftp/node_modules ``` and finally, `package.json`: ``` { "name": "sftp2sf", "version": "1.0.0", "description": "", "main": "sftp.js", "scripts": { "start": "node sftp.js", "test": "mocha", "coverage": "istanbul cover _mocha -- -R spec && open coverage/lcov-report/index.html" }, "author": "Jason Clark <jason.clark@example.com>", "license": "UNLICENSED", "dependencies": { "buffer-equal-constant-time": "^1.0.1", "byline": "^5.0.0", "eval": "^0.1.1", "fast-csv": "^2.3.0", "jsforce": "^1.7.0", "jsonminify": "^0.4.1", "minimatch": "^3.0.3", "minimist": "^1.2.0", "moment": "^2.15.1", "ssh2": "^0.5.2", "through": "^2.3.8" }, "devDependencies": { "chai": "^3.5.0", "istanbul": "^0.4.5", "mocha": "^3.1.0", "sinon": "^1.17.6", "stream-to-array": "^2.3.0" } } ``` Update: To address a proposed solution, of removing the volumes section of the docker-compose.yml: The volumes section is an important part of this setup, and allows my local host to share files with the running docker image during development, except the node_modules which only live in the image; see the linked article for full explanation. It is worth nothing that I've used this setup with those volume statements since starting on this project, and there are at least a dozen other Node modules that were installed the same way, and they are not causing the exception - if I remove `eval`, the code works. Update 17 Nov 2016: This just seemed to fix itself; after switching gears to a higher-priority project for several days, I no longer had the problem. I don't believe in bugs fixing themselves, but I cannot figure out what I may have done to resolve the issue. Today, I had to add a new node module to the project (`docker-compose -f docker-compose.yml run --rm sfsftp /bin/bash`, `npm install --save module` from inside the running shell, `docker-compose build`) and now the problem is back, only it's the new module that cannot be found.