[SOLVED] Tar with dialog not working

Forum rules
Before you post please read how to get help

[SOLVED] Tar with dialog not working

Postby RavenLX on Thu Sep 26, 2013 10:34 pm

I have this:

Code: Select all
#!/bin/bash

exec<"myfiles.txt"
while read line
do
(tar -pPrvf - $line | pv -n -s $(du -sb $line | awk '{print $1}') > myarchive.tar) 2>&1 \
| dialog --gauge "Backing up: $line" 7 70 0
done


Problem is, it shows in the dialog the file it's backing up. But, the progress bar doesn't move, things go fast and the resulting myarchive.tar is 0 bytes. The myfiles.txt contains 152 entries and some of the entries are files, and some are directories. Some directories and/or files are a bit large so they should have triggered the dialog progress bar.

What am I doing wrong here? Can anyone walk me through the correct commands?

[UPDATE] I solved it. Please see my post farther down.
Last edited by RavenLX on Fri Sep 27, 2013 7:06 pm, edited 1 time in total.
RavenLX
Level 2
Level 2
 
Posts: 78
Joined: Tue Jan 15, 2013 4:00 am

Linux Mint is funded by ads and donations.
 

Re: Tar with dialog not working

Postby RavenLX on Thu Sep 26, 2013 10:42 pm

Ideally, I would rather get the list of files with something like 'locate keyword' instead of reading a list. to get the list I did a 'locate keyword > myfiles.txt' . So if I could skip that step and do it all in the batch file, that'd be even better!
RavenLX
Level 2
Level 2
 
Posts: 78
Joined: Tue Jan 15, 2013 4:00 am

Re: Tar with dialog not working

Postby Habitual on Fri Sep 27, 2013 12:00 pm

why --progress AND pv?

Seems easier to debug if you had fewer "options" involved. but that's just my opinion.

Code: Select all
(tar -pPrvf - $line | pv -n -s $(du -sb $line | awk '{print $1}') > myarchive.tar) 2>&1
is just wrong.
tar <options> <archive.tar> <files>

IF it were me, I'd try this, or a variation on it until it mostly worked...and assuming myfiles.txt has the complete path to the files needed to backup...
Code: Select all
for i in `cat myfiles.txt` ; do tar -pPrvf  myarchive.tar $i ; done


Once that works, then I'd make that a function utillized like so:
Code: Select all
#!/bin/bash
do_work ()
{
for i in `cat myfiles.txt` ; do tar -pPrvf /path/to/myarchive.tar $i ; done
}

touch /tmp/$$
( ( echo 1 ; while [ -f /tmp/$$ ] ; do sleep 1 ; done ; echo 100 ) | dialog --gauge 7 70 0) & do_work
rm /tmp/$$
#EOF


Personally, I use yad for this stuff, and the dialog with gauge is kind of "iffy".
If you choose to try yad, here's the code I wrote for a similar routine:

Code: Select all
touch /tmp/$$
( ( echo 1 ; while [ -f /tmp/$$ ] ; do sleep 1 ; done ; echo 100 ) | yad --progress --pulsate --auto-close --text="Retrieving Snapshots..." --width=150 --title="" --undecorated --no-buttons) & do_work
rm /tmp/$$


Good luck.
What's a landing but a take off in reverse?
User avatar
Habitual
Level 8
Level 8
 
Posts: 2254
Joined: Sun Nov 21, 2010 8:31 pm
Location: LM17Q-Xfce

Re: Tar with dialog not working

Postby RavenLX on Fri Sep 27, 2013 2:06 pm

Thanks but none of these will do what I need: I take a list of files (hopefully directly from 'locate keyword' for example or a text file), tarball it while showing a progress bar or percentage of the tarball operation (so I'm not either seeing a stream of files scrolling on the screen or dots scrolling on the screen or waiting and waiting and wondering if it hung).

Your first example does not show any progress bar.
Your second example scrolls on the terminal screen (even with the echo commands removed), which is what I'm trying to avoid.
I can't find Yad in the repositories and it's not installed in my system.

BTW, I never used any --progress switch in my code?

I am still looking for a solution...
RavenLX
Level 2
Level 2
 
Posts: 78
Joined: Tue Jan 15, 2013 4:00 am

Re: Tar with dialog not working

Postby DrHu on Fri Sep 27, 2013 2:34 pm

User avatar
DrHu
Level 17
Level 17
 
Posts: 7033
Joined: Wed Jun 17, 2009 8:20 pm

Re: Tar with dialog not working

Postby RavenLX on Fri Sep 27, 2013 4:02 pm

Thanks, Dr. Hu. I already looked at that page and that's where I found out about 'dialog'. Thing is, I can't get it to work if I'm using a variable ($line).
RavenLX
Level 2
Level 2
 
Posts: 78
Joined: Tue Jan 15, 2013 4:00 am

Re: Tar with dialog not working

Postby Habitual on Fri Sep 27, 2013 5:03 pm

RavenLX wrote:Thanks but none of these will do what I need...
BTW, I never used any --progress switch in my code?


Better re-read that again. I included a snippet "....yad --progress ..." in the example I cited as my source for the snippets I gave to you to try out.
I didn't tell you to remove any echoes, nor did I say they were there to prevent accidental data loss.

What you "need" is to learn how to use tar correctly and "How do I test my tar files?"
Humor me, try
Code: Select all
cat myfiles.txt` ; do tar -pPrf  myarchive.tar $i ; done


wrt:
RavenLX wrote:myarchive.tar is 0 bytes

Test it with:
Code: Select all
tar -tf archive.tar

Does it look about correct judging by the output? based on the input myfiles.txt file?

Is the archive.tar 0 bytes? Yes, start over.
Double check your work.

Don't stick in your script and blindly expect it to work. Use it in terminal>
It's a piece of the puzzle.

Sentences like "Doesn't work" or "I can't get it to work" don't mean much.

http://www.tldp.org/LDP/abs/html/intern ... EADPIPEREF
I'm done. I fear I have spent more time on this than you have.

Good luck.
What's a landing but a take off in reverse?
User avatar
Habitual
Level 8
Level 8
 
Posts: 2254
Joined: Sun Nov 21, 2010 8:31 pm
Location: LM17Q-Xfce

Re: Tar with dialog not working

Postby RavenLX on Fri Sep 27, 2013 5:36 pm

You asked "why --progress AND pv?" At the top of your response to my first post. Therefore I thought you thought I used that when I didn't. I can't find yad in the respository.

I know you didn't "tell" me to take out the echoes from your snippets. I tried with the echoes and also without. You did give a script to try as well, and I tried that (after all I assumed it was a script since you put !#/bin/bash in the code snippet). The other stuff was tried at the terminal.

Here's what I got in the terminal when trying your snippet (I'm including the prompt as I'm copy/pasting):

Code: Select all
$ cat myfiles.txt` ; do tar -pPrf  myarchive.tar $i ; done

>

So I noted that you forgot a backtick at the start of myfiles.txt. I tried again:

Code: Select all
$ cat `myfiles.txt` ; do tar -pPrf  myarchive.tar $i ; done

bash: syntax error near unexpected token `do'

I humored you. It doesn't work. I'm sorry but none of your snippets is giving me any information that I can use to resolve my problem. As I have stated in the first post I'm looking for a progress indicator to work while tarballing files that are from a list of files (or from locate 'keyword') and not something that will scroll the screen with information.

I'm sorry if you don't want to spend more time on this. You don't have to. Thank you for trying anyway. I'll keep watching this and seeing if someone else has a solution.
RavenLX
Level 2
Level 2
 
Posts: 78
Joined: Tue Jan 15, 2013 4:00 am

[SOLVED] Re: Tar with dialog not working

Postby RavenLX on Fri Sep 27, 2013 7:05 pm

I found the problem! I want to thank all those who tried to help anyway.

Here's the solution:

Code: Select all
#!/bin/bash

IFS=$'\r\n' line=($(cat myfiles.txt))
arrlen=${#line[@]}
for (( i=1; i<${arrlen}; i++ ));
do
  (tar -pPrf myarchive.tar ${line[$i]} | pv -n -s $(du -sb ${line[$i]} | awk '{print $1}') ) 2>&1 | dialog --gauge "Backing up: ${line[$i]}" 7 70 0
done


Explaination

First off, I needed to iterate through each line in the mygiles.txt in order to get each file or directory path to feed pv, du and awk (which gets the information needed for dialog to show on the progress bar).

The second problem was putting the tar filename just before the last parenthesis (as in > myarchive.tar). Tar needs it right after the 'f' parameter.

These things fixed, it now works! My files weren't large enough though to get the progress to go past 0%. So I'm still not sure if that will work or not. But at least this is a start. Theoretically it should work.
RavenLX
Level 2
Level 2
 
Posts: 78
Joined: Tue Jan 15, 2013 4:00 am

Re: Tar with dialog not working

Postby Habitual on Fri Sep 27, 2013 9:55 pm

RavenLX wrote:You asked "why --progress AND pv?"
...
I'm sorry if you don't want to spend more time on this. You don't have to. Thank you for trying anyway.

I was short with you and I apologize. I forget that I once couldn't code my way out of a wet paper bag.

Have you tested the tars?

Code: Select all
tar -tf archive.tar | less
What's a landing but a take off in reverse?
User avatar
Habitual
Level 8
Level 8
 
Posts: 2254
Joined: Sun Nov 21, 2010 8:31 pm
Location: LM17Q-Xfce

Re: [SOLVED] Tar with dialog not working

Postby RavenLX on Fri Sep 27, 2013 10:46 pm

No problem. :) I admit even though I've grown used to the GNU/Linux command line I still don't know all that can be done with any particular command. In fact, I don't even know a fraction of all that can be done with it.

I've tested the tar in the code that didn't work and it didn't have anything in it. I tested the tar in the code that did work and everything was there as it should be.

Also, I used this code to compress the tar:

Code: Select all
(pv -n myarchive.tar | gzip -> myarchive.tgz ) 2>&1 | dialog --backtitle "$title"  --gauge "Compressing myarchive.tar" 7 70 0
if [ -f "myarchive.tgz" ]; then
  rm -rf myarchive.tar
fi


For anyone that may be reading this and wonder why I do it this way; you can't add files to a compressed tar (ie. a .tar.gz or .tgz). So I had to add them into a plain .tar. Then when done adding all the files, I can compress it with gzip and remove the tar. gzip could have done this but since we need to put this through a dialog, I used the code above instead.
RavenLX
Level 2
Level 2
 
Posts: 78
Joined: Tue Jan 15, 2013 4:00 am

Re: [SOLVED] Tar with dialog not working

Postby Habitual on Sat Sep 28, 2013 9:42 am

Well, you're making progress and that's what counts.
Scripts can always be made "better", faster, more efficient after the basic task works.

Have fun!
What's a landing but a take off in reverse?
User avatar
Habitual
Level 8
Level 8
 
Posts: 2254
Joined: Sun Nov 21, 2010 8:31 pm
Location: LM17Q-Xfce

Re: [SOLVED] Tar with dialog not working

Postby RavenLX on Sun Sep 29, 2013 11:51 pm

Here is some of what I've learned. These are code snippets that I hope will be useful to some people. Edit it to suit your needs.

Also I've found if you are concatenating stuff with directories and they show a date string of 0/0:00 in the archive, you can avoid seeing the errors by doing this:

$ tar -xzf myarchive.tgz 2>/dev/null

The archive should still extract ok.

Code: Select all
!#/bin/bash

# This is example code. It does not work as-is.
# Edit the code to suit your purpose.

clear
dialog --backtitle "$title" --infobox \
"Scanning for Files..." 3 46 1>&2
# Scan for all 'file_name' files in all of the system and add it to the list.
# This adds the entire full path.
(find / -type f ! -path "$HOME/*" ! -path "/proc/*" 2>&1  | grep file_name) > files.lst

clear
dialog --backtitle "$title" --infobox \
# Scan for user config for 'file_name and add it to the list.
# This adds the entire full path.
"Scanning for User Configuration..." 3 46 1>&2
(find $HOME/.file_name -type f) 2>&1  >> files.lst

# Back up all the files into a tar archive
clear
IFS=$'\r\n' line=($(cat files.lst))
arrlen=${#line[@]}
for(( i=0; i<${arrlen}; i++ ));
do
  clear
  (pv -n ${line[$i]} | tar pPrf myarchive.tar ${line[$i]} ) 2>&1 \
  | dialog --backtitle "$title" --gauge "Backing up file $i of $arrlen" 7 70 0
done

clear
(pv -n eclipse-backup.tar | gzip -> myarchive.tgz ) 2>&1 \
| dialog --backtitle "$title" --gauge "Compressing Archive." 7 70 0
if [ -f "myarchive.tgz" ]; then
  rm -rf myarchive.tar
fi

# Backing up one full directories into one archive
mydir="$HOME/MyDirectory"

clear
(tar cpPzf - ~/.one_of_my_config_dirs | pv -n -s $(du -sb ~/.one_of_my_config_dirs \
| awk '{print $1}') > myotherarchive.tgz) 2>&1 \
| dialog --backtitle "$title" --gauge "Backing up My Data..." 7 70 0

clear
(tar cpPzf - $mydir | pv -n -s $(du -sb $mydir \
| awk '{print $1}') > temp.tgz) 2>&1 \
| dialog --backtitle "$title" --gauge "Backing up More Data..." 7 70 0

clear
(tar -Ppz --concatenate --file=myotherarchive.tgz temp.tgz \
| pv -n temp.tgz > /dev/null) 2>&1 \
| dialog --backtitle "$title" --gauge "Compressing archive." 7 70 0
rm -rf temp.tgz
RavenLX
Level 2
Level 2
 
Posts: 78
Joined: Tue Jan 15, 2013 4:00 am

Linux Mint is funded by ads and donations.
 

Return to Scripts & Bash

Who is online

Users browsing this forum: No registered users and 0 guests