Reduce size of node_modules folder by soft linking

Issue

I have noticed running find node_modules -path '*/lodash'
that a lot of packages includes lodash, and by looking on the package.json it’s even the exact same version.

Because lodash it’s heavy (4.5MB), if all of them share the exact same memory allocation (by using a symbolic link), I should be able to reduce the size of the node_modules folder by at least 20MB, only by linking lodash modules together.

A good script should be able to find more packages rather than simply lodash, and probably save a lot more space.

Does anyone know a script/module that is able to do that?
Or I should implement it myself?

Solution

I solved it using this custom script:

import json
from itertools import groupby
from pprint import pprint
import os
import subprocess

allPackages = subprocess.check_output("find node_modules/ -name 'package.json'", shell=True).splitlines()

# Collect all modules
allModules = {}
for filePath in allPackages:
    with open(filePath.strip()) as data_file:
        data = json.load(data_file)
        packageKey = data['name'] + data['version']
        if packageKey not in allModules:
            allModules[packageKey] = list()
        allModules[packageKey].append(os.path.dirname(filePath))

# Remove not duplicated ones
for packageKey, paths in allModules.items():
    if (len(paths) > 1):
        original = allModules[packageKey].pop()
        for module in allModules[packageKey]:
            originalRelativeToModule = os.path.relpath(original,os.path.dirname(module))
            print('delete folder ', module, 'and symlink to ', originalRelativeToModule)
            print(subprocess.check_call(["rm","-rf",module]))
            print(subprocess.check_call(["ln","-s",originalRelativeToModule,module]))

Answered By – Maurizio Carboni

Answer Checked By – David Goodson (AngularFixing Volunteer)

Leave a Reply

Your email address will not be published.