Having an abundance of npm packages is a perfect example of a “bane and a boon” scenario. It often reminds me of “Amazon Forrest”. You name a problem and I bet we will find an npm package for it, and without any second thought, our first instinct would be to use it. Do you think I’m kidding? let’s try searching for “npm package to add numbers”.
I think we should always ask our self a question, why do we need this package? as we keep adding these external dependencies, our codebase will start to blot ( remember this also adds its peer dependencies ) & then we are responsible for upgrade, performance & any security vulnerability brought by these added packages.
Recently I ran into problems with npm dependency in our code-base and hence this blog is to address those mistakes and coming up with measures to mitigate the problem.
1. Know your use case :
If you notice closely ( ref: Fig 2 ) we are using a package called `flat` and then bundling it along with the code-base ( hint: used in dependencies ). So whoever consumes this service/component will have at least a `5.0.0` version of the flat package to support the use case. You can verify this by checking the lock file in the consuming app. In my case it is yarn.lock
My next instinct was to know more about the package and mainly focus on its size, peer dependency, activities, etc.
Clearly, a great, well-maintained package and actively used ( weekly downloads ) & has a size of 26.6KB. No red flags on using this package, period. But then the real question is “what is the use case” of depending on this package? so decided to look deeper into our codebase.
It was used as react-intl translation util to load the messages. But then most of our translation files are in react-intl supported JSON format.
However, we use to have a JavaScript nested object before and it clearly is a copy-paste error. How can we be so sure, here is the slimmed version of repl to simulate the exact behavior.
Refactoring and removing this flat package will save us 26.6KB in bundling size. 👏
2. Use the right package :
Pay attention and carefully add a package to your code-base. Our team is using a package called moment to convert the date format and is been added as a dependency ( ref: Fig 1 ). However, when I added this library to the consuming app, it started to throw an error about not finding moment . How could that be?
Taking a closer look at the lock file and package.jsonI realized we have unconsciously added the wrong package moment-js instead of moment .
Make sure to check/verify your package name before adding it.
3. Know your dependency:
It’s often confusing to decide whether the dependency needed just for development ( as dev dependency ) or should be carried forward (as a dependency ) so the adopting apps don’t have to worry about adding.
Anything related to testing, building, documentation, type checking, linting should never be part of dependencies but devDependencies.
Making use of peer Dependencies to indicate the consuming app to use the required compatible versions. So In our example, we can make use of this to add react, react-dom, react-intl to be part of the devDependencies and peerDependencies. This makes the hosting application to take responsibility for providing the compatible versions.
Finally, dependencies should be free from all these and should only have packages needed for the code to run. Also spending some time to understand the dependency list between hosting app ( consumer ) vs libraries dependency ( provider ) list to avoid duplications, bundle-size, etc.
Also, it’s worth reading Two React Tips from Dan Abramov.
4. Don’t be outdated :
The team put an effort to build a new react library to host re-usable components. It starts to embrace the latest / greatest packages & after thorough testing, we decided to use components from this library in our apps. But then we ran into incompatibility issues with our apps.
As you might have noticed our library is using the latest react-intl where our app didn’t get updated version since 2.9.0 ( released almost a year ago ). so it definitely doesn’t have support for useIntl hooks we have used in our library. Below is the example of such Error while integrating.
ERROR: AplicationManager.js:50 Uncaught TypeError: (0 , _reactIntl.useIntl) is not a functionUpdating the app to have 5.8.2 the version didn’t exactly solve all our problems. At some point in time between these versions react-intl had released breaking changes, so that happened next.
ERROR: in ./src/main/app/util/translation/TranslationUtil.js
Module not found: Error: Can't resolve 'react-intl/locale-data/en' in '/main/src/main/app/util/translation'After the 3.X version react-intl doesn’t ship locale-data in the bundle & here is the reference doc for this breaking API change. Now the app needs to be updated to take care of this stale code.
There is no GOLDEN RULE how often we need to update to the latest packages but in one of the podcasts, Yoni Goldberg recommends waiting for 60–90 days before embracing the latest to avoid any major security vulnerabilities 🐞.
5. Troubleshoot with .lock file :
Lock files are a way to lock down package versions and it’s all transitive dependencies. This way when we build an app across different machines we will have the same behavior. Typically top-level lock files will be referred while installing the dependency tree. Read more about yarn.lock here. Let’s understand by looking at the `package.json` of the consuming app and then its auto-generated lock file.
Our analytics app has locked `online-learning-translations` package version to 0.0.9, which makes it tightly coupled. However, it is also using a pre-production package version from `online-learning-components` which has a transitive dependency of 0.1.5 ( also locked to this version ) from online-learning-translations. Now our analytics app lock file has to please both of these versions.
Now that we understand the need for a lock file, we could make use of it while troubleshooting the dependency problem if any due to version incompatibility and/or missing dependency, etc.
Hint: Check-in your lock file to your repo.
6. Review :
Almost all of these mistakes should have been caught during development and/or in code reviews. Perhaps as a team, we need to make a cautious effort to educate ourselves and share knowledge so as a team we can mitigate the dependency problem.
It’s often emphasized we shouldn’t “ reinvent the wheel ” rather make use of thoroughly tested open source packages. So as developers we can put our effort & focus making our software better. But then that doesn’t mean we should be ignorant.
Extras 🍪:
NPM Versioning:
- Package versioning semantics: “major.minor.patch”
- Installing exact package version: “ 5.7.1”
- Installing any version matching major and minor: “ ~5.7.1” *
- Installing any version matching major version: “ ^ 5.7.1” **
- Installing any version less than or equal: “ ≤5.7.1”
- Installing any version greater than or equal: “ ≥ 5.7.1”
* patch version will keep updating
** minor & patch version will keep updating
NPM Stats:
- Most depended upon packages: list
- Total number of `react` downloads until 2020–09–24: 790,314,118
- Use `npm publish — tag beta` to publish the beta versioned package.
- More about peerDependencies here
