Split text file by line and rename based on string content

awksplit

I have a WINDOWS text file that contains the following in its own directory: It is called "test.txt"

Example Source File – test.txt:

hxxp://url.xxx/XX/file 1.txt
hxxp://url.xxx/XX/file 2.avi
hxxp://url.xxx/XX/file 3.mpg

The content of the first file would be:

hxxp://url.xxx/file 1.txt

The name of this file would be:

file 1.txt

My goal is to create separate text files containing each URL and named after the "file" part of the URL. So the first one would be "file 1.txt.txt" then "file 2.avi.avi" and so on. Split does this perfectly EXCEPT for naming the actual files.

Best Answer

This is easily done in awk. You just need to set the field separator to / and then print each line into a file whose name is the last field. Since the number of fields in awk is saved in the NF variable, you can get the last fields with $NF. Then, if you want all of them to have the txt extension, all you need is:

awk -F'/' '{print > $NF".txt"}' file

If, as seems to be the case based on your comments, your input file has Windows line endings, you need to remove the \r first:

sed 's/\r//' file | awk -F'/' '{print > $NF".txt"}'

Related Solutions

Linux – How to Split a File into Separate Files Based on Column Headers

Perl script.

Set the file name in $in in place of genome.txt or give the name as argument.

Name the script counter.pl and give it executable rights, and run it as ./counter.pl

chmod 755 counter.pl
./counter.pl

or alternatively

chmod 755 counter.pl
./counter.pl genome.txt

counter.pl:

#!/usr/bin/perl

use strict;
use warnings;

my $in = $ARGV[0] || 'genome.txt'; # input file name

open (my $F, '<', $in) or die "Cannot open input file $!";
my $n = 0;
my %fd = ();
my @fd = ();

while (<$F>) {
        # trim
        s/^\s+//;
        s/\s+$//;
        next if (!$_); # Skip empty lines
        my @x = split(/\s+/, $_);
        # 1st line, open files
        if ( ! $n++)  {
           my $fd = 0;
           for (@x) {
              open ($fd{$_}, '>', "output$_.txt") 
                or die ("Cannot open file $!")
                  if (!exists($fd{$_}));
              $fd[$fd++] = $_;
           }
        }
        else { # Write data
           die ("Should have " . ($#fd+1) . " entries on line $n")
             if ($#x != $#fd);
           for (0 .. $#x) {
              print {$fd{$fd[$_]}} ($x[$_]);
           }
           print {$fd{$_}} ("\n") for (keys %fd);
        }
}

close $fd{$_} for (keys %fd);
close $F;
# the end

Fixed the numbers of words per line (was sometimes 32, sometimes 33 in the example).

This version can accommodate any variation of columns, but all lines have to have the same numbers of words. An error will occur (the die lines) if the number of words is different, or if it cannot open files.

Just adjust the file name ($in).

Input file: (removed the extra 0 near the end)

1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4 4 30 30 30 30
0 2 2 0 2 0 2 0 2 0 2 2 0 0 2 2 2 0 1 1 1 2 0 2 0 0 0 2 0 2 0 2
0 2 1 0 1 0 1 1 1 0 2 2 0 0 2 2 2 0 0 0 0 2 0 2 0 0 1 2 0 2 0 2
0 2 1 0 1 0 1 1 1 0 2 2 0 0 2 2 2 0 0 0 0 2 0 2 0 0 1 2 0 2 0 2

output1.txt

02202020
02101011
02101011

output2.txt

2022002
1022002
1022002

output30.txt

0202
0202
0202

output3.txt

220111
220000
220000

output4.txt

2020002
2020012
2020012

Best Answer

Related Solutions

Linux – How to Split a File into Separate Files Based on Column Headers

Related Question