[True Story] The Long Battle with GitHub "file size limit" Errors ~Cut off history that even BFG can't solve with filter-branch~
August 10, 2025 development notes Github
DOCUMENTING MY BATTLE WITH THE GITHUB FILE SIZE LIMIT ERROR: WHEN BFG FAILS, FILTER-BRANCH CUTS THROUGH HISTORY
August 10, 2025
Hello! When developing web applications, you can sometimes encounter unexpected Git errors. This time, I've compiled a blog post documenting my struggle with a GitHub file size limit error.
THE BEGINNING: THE NIGHTMARE OF "EXCEEDING GITHUB'S FILE SIZE LIMIT"
One day, when I tried to push changes to a Next.js project, I received an unfamiliar error message:
remote: error: File
The cause was simple and clear. I had forgotten to add a .gitignore file to my Next.js project directory, and had committed the entire huge node_modules folder.
"See, that's easy!" I thought, so I quickly added a .gitignore file and ran the standard command to clear the cache.
Add /node_modules to .gitignore
Delete the cache and recommit
git rm -r --cached . git add . git commit -m "fix: add .gitignore"
However, even after pushing again, I got the same error. Once a "large file commit" is recorded in Git history, it doesn't disappear so easily.
ROUND 2: THE RECOMMENDED TOOL "BFG" DOESN'T WORK...?
Deciding to embark on a journey of historical revision, I decided to use a dedicated tool called BFG Repo-Cleaner, officially recommended by Git.
-
Prepare Java and download BFG.
-
Create a mirror clone for safety.
git clone --mirror https://github.com/
/ .git -
Run BFG!
But now the real battle begins.
Attempt 1: Delete by folder name → Failed!
$ java -jar ../bfg-x.x.x.jar --delete-folders node_modules
BFG aborting: No refs to update - no dirty commits found??
Attempt 2: Unprotect the latest commit → Failed!
$ java -jar ../bfg-x.x.x.jar --delete-folders node_modules --no-blob-protection
BFG aborting: No refs to update - no dirty commits found??
Attempt 3: Delete by file size → Failed!
$ java -jar ../bfg-x.x.x.jar --strip-blobs-bigger-than 100M
Warning: No large blobs matching criteria found in packfiles
Attempt 4: Optimize the database with git gc and try again → Failed!
$ git gc $ java -jar ../bfg-x.x.x.jar --strip-blobs-bigger-than 100M
Warning: No large blobs matching criteria found in packfiles
Attempt 5: Specify filename directly → Failed!
$ java -jar ../bfg-x.x.x.jar -D
BFG aborting: No refs to update - no dirty commits found??
No matter what I tried, BFG couldn't find the large file and aborted the process. Perhaps it was because it was in a subdirectory or perhaps the repository was in a special state... I'm not sure why, but I had no choice but to give up on using BFG.
FINAL BATTLE: THE FORBIDDEN GIT FILTER-BRANCH
There was only one option left: Git's built-in ultimate weapon, git filter-branch. This command is very powerful, but it also carries the risk of destroying your repository.
[Most Important] Always back up your entire project folder before proceeding!
Prepared for the worst, I executed the following command:
- REWRITING HISTORY
This scans the entire history of the repository and deletes the specified folder (in this case,
git filter-branch --force --index-filter 'git rm -r --cached --ignore-unmatch <project_name>/node_modules' --prune-empty --tag-name-filter cat -- --all
- REPOSITORY CLEANUP
Completely delete the backup data left behind by filter-branch and optimize your repository.
Remove backup references (delete the folder directly because an error occurred in Windows PowerShell)
For PowerShell:
Remove-Item -Recurse -Force .git\refs\original
For Linux/macOS/Git Bashの場合：
rm -rf .git/refs/original/
Remove reference log and unnecessary data
git reflog expire --expire=now --all git gc --prune=now
- FORCE PUSH
Force push to GitHub, assuming the local revision history is correct.
git push origin --force --all git push origin --force --tags
When this push went through, there was a progress bar signaling the end of a long battle.
SUMMARY
-
Include node_modules in .gitignore from the beginning! (Basic, but most important)
-
If you've committed a large file to your Git history, you'll need to rewrite your history.
-
To rewrite your history, it's safe and easy to first try BFG Repo-Cleaner.
-
In the rare cases where BFG doesn't work, git filter-branch is a last resort.
-
Always back up your changes before performing risky operations like filter-branch or push --force!
I hope this article will help someone struggling with the same error.