Unlimited history for bash shell
misc bash shell command-line
This post is intended as a guide to increase the size of bash history (or even better, configure it to have an unlimited size). We will try to understand the different environment variables involved and tweak them to have a bigger(or unlimited) history.
Why do we need more history?
As developers, we find ourselves trying to remember a certain command we used in the past (which is pretty lengthy to
type or uses a specific combination of options/arguments).
The reverse history search (ctrl-r
) feature comes in very handy in such situations. This command lets us search through the history of commands stored in the
history file.
By default, the number of commands persisted in the history is just 500
- This is insufficient, compared to the number of commands we type on a typical day.
Once this limit is hit, older commands get truncated as newer commands are added to the history file. I verified that I type around 100 commands a day, which means I’ll only retain 5-days worth of commands in my history and lose everything before that.
By increasing the size of our history, more commands can be retained and be available for searching. This way, we can
search for commands used months(or years) ago.
Understanding related environment variables
Before we get started, let’s try to understand a few environment variables pertaining to history in bash.
HISTFILESIZE
- Refers to the maximum number of commands to store in the bash history file (Default is 500
)
HISTSIZE
- Refers to the maximum number of commands to load into memory from the history file (Default is 500
)
HISTFILE
- Refers to the path/location of the history file (Default is ~/.bash_history
)
These are the variables we will be tweaking for a bigger history.
You can find more details about these variables from the man page for bash
(Try man bash
from the commandline).
Steps to setup bash to have unlimited history
1. Change the bash history file
As documented in the StackOverflow post, there could be issues with the default bash history file getting truncated in certain cases. We can
workaround this problem by modifying the HISTFILE
env variable to use a different file for storing history.
# In ~/.bashrc
export HISTFILE=~/.history
2. Increase history size
Setting the HISTFILESIZE
and HISTSIZE
variables to an empty string makes the bash history size unlimited.
# In ~/.bashrc
export HISTFILESIZE=
export HISTSIZE=
In bash 4.3 or later, you can set these to -1 as well, to have the safe effect. (Rather than setting it to UNLIMITED
), some set this to a really high value (100000
), so that the history file doesn’t grow indefinitely.
3. Copy existing history
Though we start using a different history file, we shouldn’t lose commands already on the old history file. Let’s just copy the contents of the old file to new one.
# On the command line
cp ~/.bash_history ~/.history
Some nice-to-haves
Now that we have increased the size of our history, let’s try to make it more robust (And that means meddling with more environment variables!).
1. Immediate append
PROMPT_COMMAND="history -a; $PROMPT_COMMAND"
PROMPT_COMMAND
- Setting this to history -a
flushes the command to the history file immediately (otherwise, this
would happen only when the shell exits, and you could lose history upon unexpected/unclean termination of the shell).
2. Add Timestamp to history
# In ~/.bashrc
export HISTTIMEFORMAT="[%F %T] "
HISTTIMEFORMAT
- Setting this ensures timestamps are written to the history file (along with the actual command
). You can later find out the exact time of execution for a command, if needed. Also, using the history
command
to view the last n commands displays the timestamp information, in addition to the actual command:
# On the command line
~$ history 5
9626 [2019-12-03 21:51:02] cd ~
9627 [2019-12-03 21:51:04] ls -lrt
9628 [2019-12-03 21:51:08] df -h
9629 [2019-12-03 21:51:12] man bash
9630 [2019-12-03 21:51:16] history 5
3. Handling duplicate commands
(While searching with Ctrl+R
) Stepping through the history with UP and DOWN keys becomes a bit annoying if the same
command comes up again and again. A better option is to skip duplicates and show each command only once with the
following setting. (Duplicate commands are still written to the history - However, they come up just once while
stepping
through).
# In ~/.bashrc
export HISTCONTROL=ignoredups
We can take it up a notch by not writing duplicates to the history file at all, using the erasedups
option.
# In ~/.bashrc
export HISTCONTROL=erasedups
The ensures all previous lines matching the current command is removed from history before the current command is saved.
4. Sharing history
When we have multiple bash shells open simultaneously, a command entered in one shell can be made available to all other shells with the following:
PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"
As we know already, history -a
appends the command to history file immediately. Additionally, history -c
clears
the in-memory history and history -r
appends the contents history file to the in-memory history. (Together
, history -c
+ history -r
gives the effect of refreshing the in-memory history with the updated history file, after every
command). I personally don’t have this enabled since there is not much value add, and refreshing history after
every command can get expensive as our history file grows in size.
Putting everything together
# In ~/.bashrc
export HISTFILESIZE=
export HISTSIZE=
export HISTFILE=~/.history
export HISTTIMEFORMAT="[%F %T] "
PROMPT_COMMAND="history -a; $PROMPT_COMMAND"
export HISTCONTROL=ignoredups
# On the command line
cp ~/.bash_history ~/.history
Conclusion
In this post, we saw how to make our bash history bigger, so that searching for past commands is seamless. Personally, I didn’t think there was so much to bash history(Boy, was I wrong!), and learnt a lot of things while setting this up - I hope you found the post useful too. See you in the next post.