Ubuntu – How to combine multiple text files into one text file ordered by date created

14.04command linedropboxoutputtext;

I'm a newbie so please help:

I keep a journal on my iPhone using Scratch that outputs all notes I make to a separate .txt file stored in Dropbox.

I have this synced with my Ubuntu 14.04 system, so in my Files I have a folder with all the text files stored in it here:

/home/stuart/Dropbox/Scratch

I want to run a command that concatenates all these files into one file, with the following provisos:

  1. Ordered by date created (earliest file first)
  2. Print the date of the file on a separate line before the contents of the file
  3. Include a blank line followed by some kind of separator line after each file

So the output file has entries that look something like this:

12-01-2014 11:01 AM:
A coffee shop in Israel. The sign outside reads:

"Coffee" – 9 shekel

"Coffee please" – 8 shekel

"Good morning, could I have a coffee please?" – 7 shekel


25-01-2014 11:01 AM:

You cannot outperform your ego – ole Gunnar solskjaer

And so on. I used to use a different app that did this kind of auto append but I have no idea how to replicate it.

I've looked over a lot of the help files on here but I've not found any that can help with the output I have in mind.

Any help greatly appreciated!


MORE INFO

I tried creating the script suggested below and followed the steps. However I get this response:

stuart@StudioClough:/home$ chmod +x $HOME/my_concat

stuart@StudioClough:/home$ ./my_concat /home/stuart/Dropbox/Scratch > new_concatenated_file

bash: new_concatenated_file: Permission denied

Do I have to somehow run it as sudo?

Best Answer

It can be done with a python script, with one sidenote: I took the modification date instead of the creation date, since the creation date will almost certainly not match the real creation date: it is the date the file was copied to the computer, while the modification date seems unchanged during copying (see discussion at @cOrps answer). You will have to see if it works in your situation.

If that is acceptable for you, you can use the script below to create a combined file with your notes. It reads the notes, sorts them and appends them to a text file (creates it if it doesn't exist).

The good news is that you can append your new notes to the same file without overwriting the old ones.

Example output:

Mon Sep 29 08:48:31 2014
This is my first note.
As you can read, I am not really awake yet.

----------
Mon Sep 29 09:04:06 2014
It is really time I am going to eat something.
I am a bit hungry.
Making it a bit longer.

----------

How to use:

  • Copy the script below into an empty file, save it as add_notes.py
  • change the directories for files_dir (where your notes are) and the file in which you want to save the notes: combined_file (the script creates the file if it doesn't exist)
  • run the script in a terminal window by typing the command:

    python3 /path/to/add_notes.py
    

The script:

#!/usr/bin/env python3

import os
import time
import subprocess

# --------------------------------------------------------
files_dir = "/path/to/your/textfiles"
combined_file = "/path/to/your/combined/file.txt"
# ---------------------------------------------------------
notes = []

if not os.path.exists(combined_file):
    subprocess.Popen(["touch", combined_file])

def read_file(file):
    with open(file) as note:
        return note.read()

def append_file(combined_file, text):
    with open(combined_file, "a") as notes:
        notes.write(text)

for root, dirs, files in os.walk(files_dir):
    for name in files:
        subject = root+"/"+name
        cr_date_text = time.ctime(os.path.getmtime(subject))
        cr_date_n = os.stat(subject).st_mtime
        notes.append((cr_date_n, cr_date_text, subject))

notes.sort(key=lambda x: x[0])

for note in notes:
    text = note[1]+"\n"+read_file(note[2])+"\n"+"-"*10+"\n"
    append_file(combined_file, text)