ERROR: new collation (en_US.UTF-8) is incompatible with the collation of the template database (C)

I was getting this error when installing postgresql:
ERROR: new collation (en_US.UTF-8) is incompatible with the collation of the template database (C)
HINT: Use the same collation as in the template database, or use template0 as template.

The situation was I was trying to install packages automatically (via script) after a machine boots.
The problem was the locale that is set is POSIX and the database needed en_US.UTF-8.

I need to do the following prior to package installation:
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_ALL=en_US.UTF-8