llms.txt: Broken Accents, Three Bytes to Fix Them

If your llms.txt shows “é” instead of “é”, it is served as text/plain with no character set, and the browser guesses wrong. The fastest fix: save the file as UTF-8 with a BOM. The cleaner fix: tell the server the file is UTF-8.

The symptom

The file is written correctly, in UTF-8, but once online the accents come out wrong. You see it right away by opening the address in a browser.

The cause can be checked with one command:

curl -sI https://mysite.com/llms.txt | grep -i content-type
# Content-Type: text/plain

No charset=utf-8: the server specifies nothing, so the browser (or the bot reading the file) picks an encoding by itself.

Fix 1: the BOM, in one minute

The BOM (byte order mark) is a three-byte signature, EF BB BF, placed at the start of the file. It tells the tools reading it “this file is UTF-8”.

With Notepad++:

1. Open the file, then theEncoding menu.
2. ChooseUTF-8-BOM. Don’t pick “Convert to ANSI”, or “Encode in…”.
3. Save. The status bar at the bottom right should show “UTF-8-BOM”.
4. Upload the file by FTP to the site root, inbinary mode so the software doesn’t touch it.
5. Reload withCtrl+F5: browsers like to keep the old version in cache.

I did this on my file: the server still answers text/plain with no character set, but the browser now reads the UTF-8 encoding and the accents are correct.

Fix 2: tell the server, with .htaccess

On an Apache server, one rule is enough, placed at the very top of the .htaccess file:

<Files "llms.txt">
  ForceType "text/plain; charset=utf-8"
</Files>

Two precautions:

• Make abackup copy of .htaccess before touching it. A mistake causes a 500 page on the whole site.
• Save it asUTF-8 without BOM: in an .htaccess, a BOM triggers that same 500 error.

This is the lasting fix, since the file no longer needs a BOM. I haven’t applied it on my site, so I haven’t tested it online. After the change, the same curl -sI command should show charset=utf-8.

Why bother

An llms.txt is meant for machines, and nothing guarantees they correct a doubtful encoding. The fix takes two minutes: better to have the file readable as is.

Key takeaway

Check the header with curl -sI. If it shows no character set, add a BOM to go fast, or an .htaccess rule to do it properly, then reread the address after a forced reload.