Write a script to edit a txt file ?

About writing shell scripts and making the most of your shell
Forum rules
Topics in this forum are automatically closed 6 months after creation.
Locked
Bad_Dream

Write a script to edit a txt file ?

Post by Bad_Dream »

im trying to figure out how to edit trivia questions to change them from one format to another
the questions are in this format :
spring:What season is the setting for Shakespeare's A Midsummer Nights Dream ?
and i need them in this format:
What season is the setting for Shakespeare's A Midsummer Nights Dream ?*spring

i have no idea where to start to write a script to do this and was hoping someone could point me in the right direction! thanks :D
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
User avatar
Oscar799
Level 20
Level 20
Posts: 10396
Joined: Tue Aug 11, 2009 9:21 am
Location: United Kingdom

Re: Write a script to edit a txt file ?

Post by Oscar799 »

Moved here by moderator
Image
BrianD

Re: Write a script to edit a txt file ?

Post by BrianD »

There's an O'Reilly book you might find quite useful, if you do this sort of thing often -- Regular Expressions.

maybe something like

Code: Select all

/^(.*):(.*)$/$2\*$1/
User avatar
xenopeek
Level 25
Level 25
Posts: 29501
Joined: Wed Jul 06, 2011 3:58 am

Re: Write a script to edit a txt file ?

Post by xenopeek »

To use the regular expression, you could use the command line tool called sed. Assuming you have an input file called "input", and the regular expression is correct, you would run the following command:

Code: Select all

sed -r 's/^(.*):(.*)$/\2*\1/' input > output
This will create a file called "output", where the switch should have been done for each matching line.

Ninja edit: to learn about sed, consult its manpage (run "man sed" on the terminal), or have a look at the classic "sed one liners" list of examples: http://sed.sourceforge.net/sed1line.txt
Image
BrianD

Re: Write a script to edit a txt file ?

Post by BrianD »

...or, use perl inline:

Code: Select all

perl -pi -e 's/^(.*)\:(.*)$/$2\*$1/g' triv1.txt
...that'll reformat the file (triv1.txt), per your specifications, in place -- no need to create a new file. :D
wei2912

Re: Write a script to edit a txt file ?

Post by wei2912 »

Bad_Dream wrote:im trying to figure out how to edit trivia questions to change them from one format to another
the questions are in this format :
spring:What season is the setting for Shakespeare's A Midsummer Nights Dream ?
and i need them in this format:
What season is the setting for Shakespeare's A Midsummer Nights Dream ?*spring

i have no idea where to start to write a script to do this and was hoping someone could point me in the right direction! thanks :D
Since Bad_Dream is a newbie, I've decided not to use regular expressions, but a while loop.

Pros of while loop
- Writes to the same file - can be executed many times for a group of trivia files
- Simple to write
Cons of for loop
- Slower than regular expressions

Pros of regular expressions
- Edits the file directly, does not generate a new one
- Faster than for loop
Cons of regular expressions
- Not as easy to write.

I'll start off with an explanation and a commented script.

First, we run a big while loop. In this while loop, we assign variable 'string' to the current line of the file. Then, the position of substring ":" is found. The substrings are then extracted. With this each line can be rearranged to obtain the result.

Code:

Code: Select all

#!/bin/bash
FILENAME="trivia1.txt"

while read string
do
    # Start of line parsing.
    echo $string # Prints out the string.

    i=`expr index "$string" :` # Obtain index of :

    # Extracts a substring of position x and position x + 1, or the letter at position x
    # Checks if equal to a colon.
    answer=${string:0:$((i-1))} # Obtains everything before the colon
    question=${string:$i} # Obtains everything after the colon

    echo "$answer"
    echo "$question"

    result=`echo $question*$answer`
    echo $result
    echo $result >> ./result.txt
done < $FILENAME # Reads the file.

echo ""
echo "File conversion has finished. Check result.txt in this directory."
echo "Press enter to exit"
read enter # Just a goodie, no need to remember this
Now, in the terminal, run (make sure you are in the correct directory):

Code: Select all

wei2912@ubuntu-desktop ~ $ chmod +x ./bash.sh
wei2912@ubuntu-desktop ~ $ ./bash.sh
testing:Testing question
testing
Testing question
Testing question*testing

File conversion has finished. Check result.txt in this directory.
Press enter to exit

wei2912@ubuntu-desktop ~ $ 

Try running this script with a file trivia1.txt containing answers and questions. Make sure there is an endline after the last setence for this script to work properly.

If you're interested in editing this script, contact me, else simply change the variable FILENAME to what the name of your file is.

This script is extremely efficient, it can convert 4500 lines in 8 seconds.
Bad_Dream

Re: Write a script to edit a txt file ?

Post by Bad_Dream »

awesome wei2912 and to everyone else that responded....gunna try this today ....this is gunna save me hourssssss of tedious editing! tyty :D
Bad_Dream

Re: Write a script to edit a txt file ?

Post by Bad_Dream »

theres just one problem....the strings dont stay on the same line and for the life of me i cant figure out how to make it stay on the same. ive tried echo -ne and the printf command and managed to get it to stay on the say line with printf but it truncated the strings to the first word...completely baffled that something so simple is turning out to be so hard. does anyone have any ideas?
User avatar
xenopeek
Level 25
Level 25
Posts: 29501
Joined: Wed Jul 06, 2011 3:58 am

Re: Write a script to edit a txt file ?

Post by xenopeek »

Give us a few input lines, like 10 lines directly from your input file of which at least a few are causing the problems. We'll probably can offer a better solution having some actual input.
Image
Bad_Dream

Re: Write a script to edit a txt file ?

Post by Bad_Dream »

okeedokee heres the input file im testing with atm


turner:What English artist had the forenames Joseph Mallord William ?
black:What colour was the Maltese Falcon ?
troy:What city fell to the Wooden Horse ?
dormouse:Who was asleep between the March Hare and the Hatter ?
blue:What colour eyes did The Boys from Brazil have ?
shazam:What was Captain Marvel's magic word?
michelangelo:Who carved the famed Medici tombs in Florence ?
paprika:What is the Hungarian word for Pepper ?
the:What is the most used word in written English ?
achilles:Who was dipped in the River Styx ?
snoopy:What Peanuts character has a brother named Spike ?
armageddon:What Leon Uris novel deals with the Russian capture of Berlin ?
white:What colour was Moby Dick ?
eight:How many reindeer pull Santa's sleigh ?

there are 345 lines like that. i need the script to convert them to this format

What English artist had the forenames Joseph Mallord William ?*turner
What colour was the Maltese Falcon ?*black
What city fell to the Wooden Horse ?*troy
Who was asleep between the March Hare and the Hatter ?*dormouse
What colour eyes did The Boys from Brazil have ?*blue
What was Captain Marvel's magic word?*shazam
Who carved the famed Medici tombs in Florence ?*michelangelo
What is the Hungarian word for Pepper ?*paprika
What is the most used word in written English ?*the
Who was dipped in the River Styx ?*achilles
What Peanuts character has a brother named Spike ?*snoopy
What Leon Uris novel deals with the Russian capture of Berlin ?*armageddon
What colour was Moby Dick ?*white
How many reindeer pull Santa's sleigh ?*eight

the script that wei2912 wrote almost works. the output file looks like this

What English artist had the forenames Joseph Mallord William ?
*turner
What colour was the Maltese Falcon ?
*black
What city fell to the Wooden Horse ?
*troy
Who was asleep between the March Hare and the Hatter ?
*dormouse
What colour eyes did The Boys from Brazil have ?
*blue
What was Captain Marvel's magic word?
*shazam
Who carved the famed Medici tombs in Florence ?
*michelangelo
What is the Hungarian word for Pepper ?
*paprika
What is the most used word in written English ?
*the
Who was dipped in the River Styx ?
*achilles
What Peanuts character has a brother named Spike ?
*snoopy
What Leon Uris novel deals with the Russian capture of Berlin ?
*armageddon
What colour was Moby Dick ?
*white
How many reindeer pull Santa's sleigh ?
*eight


hope that clears things up
User avatar
xenopeek
Level 25
Level 25
Posts: 29501
Joined: Wed Jul 06, 2011 3:58 am

Re: Write a script to edit a txt file ?

Post by xenopeek »

Problem is $question also gets the newline from the input file. Possible solution, replace:

Code: Select all

question=${string:$i} # Obtains everything after the colon
with:

Code: Select all

question=${string:$i} # Obtains everything after the colon
question=${question%?} # Remove the newline
Or perhaps your input file was created on Windows? Then first do a dos2unix on it to replace Windows' end-of-line character sequence with Linux's end-of-line character. You can also do this in the "Save As" window in Gedit / Pluma, near the bottom you can select the "Line Ending" to use. Make sure it is set to Unix/Linux. (Saving you at least one byte per line! Linux is lean :wink:)

Anyway, learning to program BASH is useful to have in your toolbox. But perhaps the single line sed or perl with a regular expression is more handy in this case. I found I made two typos in the sed command, fixed those and tested it with your input. Should work now.
Image
Bad_Dream

Re: Write a script to edit a txt file ?

Post by Bad_Dream »

yayyyyyy thank you vincent your solution worked! tytyty
wei2912

Re: Write a script to edit a txt file ?

Post by wei2912 »

Bad_Dream wrote:yayyyyyy thank you vincent your solution worked! tytyty
Willing to give me some trivia questions for my bot's hangman game? :)

Vincent: You did a great job :D i'm not so good at bash and had a bit of trouble writing that script.
Loomx

Re: Write a script to edit a txt file ?

Post by Loomx »

Hi there,

Another option is awk.
Then you don't need so many confusing backslashes.

Awk splits the input lines into fields. The default field separator is whitespace, but for this file, set it to :
Then each line will just have 2 fields, and you just print them in reverse order.
(I'm assuming that you want the * in there too like in your example)

Code: Select all

awk -F : '{ print $2 "*"$1 }' input > output
wei2912

Re: Write a script to edit a txt file ?

Post by wei2912 »

Loomx wrote:Hi there,

Another option is awk.
Then you don't need so many confusing backslashes.

Awk splits the input lines into fields. The default field separator is whitespace, but for this file, set it to :
Then each line will just have 2 fields, and you just print them in reverse order.
(I'm assuming that you want the * in there too like in your example)

Code: Select all

awk -F : '{ print $2 "*"$1 }' input > output
The * is a delimiter to split up the question and answer :)
Locked

Return to “Scripts & Bash”