Write a script to edit a txt file ?

Forum rules
Before you post please read this

Write a script to edit a txt file ?

Postby Bad_Dream on Tue Jun 19, 2012 9:58 am

im trying to figure out how to edit trivia questions to change them from one format to another
the questions are in this format :
spring:What season is the setting for Shakespeare's A Midsummer Nights Dream ?
and i need them in this format:
What season is the setting for Shakespeare's A Midsummer Nights Dream ?*spring

i have no idea where to start to write a script to do this and was hoping someone could point me in the right direction! thanks :D
Bad_Dream
Level 1
Level 1
 
Posts: 7
Joined: Tue Jun 19, 2012 9:27 am

Linux Mint is funded by ads and donations.
 

Re: Write a script to edit a txt file ?

Postby Oscar799 on Tue Jun 19, 2012 10:01 am

Moved here by moderator
Image
"Don't fix it if it ain't broken,don't break it if you can't fix it" Husse
Registered Linux User #511789
User avatar
Oscar799
Level 18
Level 18
 
Posts: 8699
Joined: Tue Aug 11, 2009 9:21 am
Location: United Kingdom

Re: Write a script to edit a txt file ?

Postby BrianD on Tue Jun 19, 2012 10:39 am

There's an O'Reilly book you might find quite useful, if you do this sort of thing often -- Regular Expressions.

maybe something like

Code: Select all
/^(.*):(.*)$/$2\*$1/
need I say more??
BrianD
Level 4
Level 4
 
Posts: 293
Joined: Fri Feb 12, 2010 12:22 am
Location: Tallahassee, Florida, USA

Re: Write a script to edit a txt file ?

Postby xenopeek on Tue Jun 19, 2012 10:58 am

To use the regular expression, you could use the command line tool called sed. Assuming you have an input file called "input", and the regular expression is correct, you would run the following command:
Code: Select all
sed -r 's/^(.*):(.*)$/\2*\1/' input > output

This will create a file called "output", where the switch should have been done for each matching line.

Ninja edit: to learn about sed, consult its manpage (run "man sed" on the terminal), or have a look at the classic "sed one liners" list of examples: http://sed.sourceforge.net/sed1line.txt
Forum Rules | IRC Channel Rules
Image
Arch Linux / 64-bit / Gnome Shell
User avatar
xenopeek
Level 21
Level 21
 
Posts: 13683
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: Write a script to edit a txt file ?

Postby BrianD on Tue Jun 19, 2012 11:21 am

...or, use perl inline:

Code: Select all
perl -pi -e 's/^(.*)\:(.*)$/$2\*$1/g' triv1.txt


...that'll reformat the file (triv1.txt), per your specifications, in place -- no need to create a new file. :D
need I say more??
BrianD
Level 4
Level 4
 
Posts: 293
Joined: Fri Feb 12, 2010 12:22 am
Location: Tallahassee, Florida, USA

Re: Write a script to edit a txt file ?

Postby wei2912 on Tue Jun 19, 2012 9:43 pm

Bad_Dream wrote:im trying to figure out how to edit trivia questions to change them from one format to another
the questions are in this format :
spring:What season is the setting for Shakespeare's A Midsummer Nights Dream ?
and i need them in this format:
What season is the setting for Shakespeare's A Midsummer Nights Dream ?*spring

i have no idea where to start to write a script to do this and was hoping someone could point me in the right direction! thanks :D


Since Bad_Dream is a newbie, I've decided not to use regular expressions, but a while loop.

Pros of while loop
- Writes to the same file - can be executed many times for a group of trivia files
- Simple to write
Cons of for loop
- Slower than regular expressions

Pros of regular expressions
- Edits the file directly, does not generate a new one
- Faster than for loop
Cons of regular expressions
- Not as easy to write.

I'll start off with an explanation and a commented script.

First, we run a big while loop. In this while loop, we assign variable 'string' to the current line of the file. Then, the position of substring ":" is found. The substrings are then extracted. With this each line can be rearranged to obtain the result.

Code:
Code: Select all
#!/bin/bash
FILENAME="trivia1.txt"

while read string
do
    # Start of line parsing.
    echo $string # Prints out the string.

    i=`expr index "$string" :` # Obtain index of :

    # Extracts a substring of position x and position x + 1, or the letter at position x
    # Checks if equal to a colon.
    answer=${string:0:$((i-1))} # Obtains everything before the colon
    question=${string:$i} # Obtains everything after the colon

    echo "$answer"
    echo "$question"

    result=`echo $question*$answer`
    echo $result
    echo $result >> ./result.txt
done < $FILENAME # Reads the file.

echo ""
echo "File conversion has finished. Check result.txt in this directory."
echo "Press enter to exit"
read enter # Just a goodie, no need to remember this


Now, in the terminal, run (make sure you are in the correct directory):
Code: Select all
wei2912@ubuntu-desktop ~ $ chmod +x ./bash.sh
wei2912@ubuntu-desktop ~ $ ./bash.sh
testing:Testing question
testing
Testing question
Testing question*testing

File conversion has finished. Check result.txt in this directory.
Press enter to exit

wei2912@ubuntu-desktop ~ $



Try running this script with a file trivia1.txt containing answers and questions. Make sure there is an endline after the last setence for this script to work properly.

If you're interested in editing this script, contact me, else simply change the variable FILENAME to what the name of your file is.

This script is extremely efficient, it can convert 4500 lines in 8 seconds.
wei2912
Level 2
Level 2
 
Posts: 79
Joined: Thu Jun 02, 2011 1:00 am

Re: Write a script to edit a txt file ?

Postby Bad_Dream on Wed Jun 20, 2012 9:13 am

awesome wei2912 and to everyone else that responded....gunna try this today ....this is gunna save me hourssssss of tedious editing! tyty :D
Bad_Dream
Level 1
Level 1
 
Posts: 7
Joined: Tue Jun 19, 2012 9:27 am

Re: Write a script to edit a txt file ?

Postby Bad_Dream on Thu Jun 21, 2012 12:49 pm

theres just one problem....the strings dont stay on the same line and for the life of me i cant figure out how to make it stay on the same. ive tried echo -ne and the printf command and managed to get it to stay on the say line with printf but it truncated the strings to the first word...completely baffled that something so simple is turning out to be so hard. does anyone have any ideas?
Bad_Dream
Level 1
Level 1
 
Posts: 7
Joined: Tue Jun 19, 2012 9:27 am

Re: Write a script to edit a txt file ?

Postby xenopeek on Thu Jun 21, 2012 12:57 pm

Give us a few input lines, like 10 lines directly from your input file of which at least a few are causing the problems. We'll probably can offer a better solution having some actual input.
Forum Rules | IRC Channel Rules
Image
Arch Linux / 64-bit / Gnome Shell
User avatar
xenopeek
Level 21
Level 21
 
Posts: 13683
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: Write a script to edit a txt file ?

Postby Bad_Dream on Thu Jun 21, 2012 2:13 pm

okeedokee heres the input file im testing with atm


turner:What English artist had the forenames Joseph Mallord William ?
black:What colour was the Maltese Falcon ?
troy:What city fell to the Wooden Horse ?
dormouse:Who was asleep between the March Hare and the Hatter ?
blue:What colour eyes did The Boys from Brazil have ?
shazam:What was Captain Marvel's magic word?
michelangelo:Who carved the famed Medici tombs in Florence ?
paprika:What is the Hungarian word for Pepper ?
the:What is the most used word in written English ?
achilles:Who was dipped in the River Styx ?
snoopy:What Peanuts character has a brother named Spike ?
armageddon:What Leon Uris novel deals with the Russian capture of Berlin ?
white:What colour was Moby Dick ?
eight:How many reindeer pull Santa's sleigh ?

there are 345 lines like that. i need the script to convert them to this format

What English artist had the forenames Joseph Mallord William ?*turner
What colour was the Maltese Falcon ?*black
What city fell to the Wooden Horse ?*troy
Who was asleep between the March Hare and the Hatter ?*dormouse
What colour eyes did The Boys from Brazil have ?*blue
What was Captain Marvel's magic word?*shazam
Who carved the famed Medici tombs in Florence ?*michelangelo
What is the Hungarian word for Pepper ?*paprika
What is the most used word in written English ?*the
Who was dipped in the River Styx ?*achilles
What Peanuts character has a brother named Spike ?*snoopy
What Leon Uris novel deals with the Russian capture of Berlin ?*armageddon
What colour was Moby Dick ?*white
How many reindeer pull Santa's sleigh ?*eight

the script that wei2912 wrote almost works. the output file looks like this

What English artist had the forenames Joseph Mallord William ?
*turner
What colour was the Maltese Falcon ?
*black
What city fell to the Wooden Horse ?
*troy
Who was asleep between the March Hare and the Hatter ?
*dormouse
What colour eyes did The Boys from Brazil have ?
*blue
What was Captain Marvel's magic word?
*shazam
Who carved the famed Medici tombs in Florence ?
*michelangelo
What is the Hungarian word for Pepper ?
*paprika
What is the most used word in written English ?
*the
Who was dipped in the River Styx ?
*achilles
What Peanuts character has a brother named Spike ?
*snoopy
What Leon Uris novel deals with the Russian capture of Berlin ?
*armageddon
What colour was Moby Dick ?
*white
How many reindeer pull Santa's sleigh ?
*eight


hope that clears things up
Bad_Dream
Level 1
Level 1
 
Posts: 7
Joined: Tue Jun 19, 2012 9:27 am

Re: Write a script to edit a txt file ?

Postby xenopeek on Thu Jun 21, 2012 3:08 pm

Problem is $question also gets the newline from the input file. Possible solution, replace:
Code: Select all
question=${string:$i} # Obtains everything after the colon

with:
Code: Select all
question=${string:$i} # Obtains everything after the colon
question=${question%?} # Remove the newline

Or perhaps your input file was created on Windows? Then first do a dos2unix on it to replace Windows' end-of-line character sequence with Linux's end-of-line character. You can also do this in the "Save As" window in Gedit / Pluma, near the bottom you can select the "Line Ending" to use. Make sure it is set to Unix/Linux. (Saving you at least one byte per line! Linux is lean :wink:)

Anyway, learning to program BASH is useful to have in your toolbox. But perhaps the single line sed or perl with a regular expression is more handy in this case. I found I made two typos in the sed command, fixed those and tested it with your input. Should work now.
Forum Rules | IRC Channel Rules
Image
Arch Linux / 64-bit / Gnome Shell
User avatar
xenopeek
Level 21
Level 21
 
Posts: 13683
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: Write a script to edit a txt file ?

Postby Bad_Dream on Thu Jun 21, 2012 3:32 pm

yayyyyyy thank you vincent your solution worked! tytyty
Bad_Dream
Level 1
Level 1
 
Posts: 7
Joined: Tue Jun 19, 2012 9:27 am

Re: Write a script to edit a txt file ?

Postby wei2912 on Wed Jun 27, 2012 4:12 am

Bad_Dream wrote:yayyyyyy thank you vincent your solution worked! tytyty


Willing to give me some trivia questions for my bot's hangman game? :)

Vincent: You did a great job :D i'm not so good at bash and had a bit of trouble writing that script.
wei2912
Level 2
Level 2
 
Posts: 79
Joined: Thu Jun 02, 2011 1:00 am

Re: Write a script to edit a txt file ?

Postby Loomx on Sat Jul 28, 2012 7:05 am

Hi there,

Another option is awk.
Then you don't need so many confusing backslashes.

Awk splits the input lines into fields. The default field separator is whitespace, but for this file, set it to :
Then each line will just have 2 fields, and you just print them in reverse order.
(I'm assuming that you want the * in there too like in your example)

Code: Select all
awk -F : '{ print $2 "*"$1 }' input > output
Loomx
Level 1
Level 1
 
Posts: 1
Joined: Sat Jul 28, 2012 6:52 am

Re: Write a script to edit a txt file ?

Postby wei2912 on Mon Jul 01, 2013 9:33 am

Loomx wrote:Hi there,

Another option is awk.
Then you don't need so many confusing backslashes.

Awk splits the input lines into fields. The default field separator is whitespace, but for this file, set it to :
Then each line will just have 2 fields, and you just print them in reverse order.
(I'm assuming that you want the * in there too like in your example)

Code: Select all
awk -F : '{ print $2 "*"$1 }' input > output


The * is a delimiter to split up the question and answer :)
wei2912
Level 2
Level 2
 
Posts: 79
Joined: Thu Jun 02, 2011 1:00 am

Linux Mint is funded by ads and donations.
 

Return to Scripts & Bash

Who is online

Users browsing this forum: No registered users and 1 guest