migrating files using tar.gz

Chat about anything related to Linux Mint

migrating files using tar.gz

Postby mike acker on Wed Feb 19, 2014 8:23 pm

it appears from my experiments that if i make a .tar archive on an older system and then un-pack that archive on a newer system -- thus "migrating the files" that the chmod permissions on the newer system end up as expected: me, as owner, same as when I create a file and save it, e.g. using gedit or LibreOffice.

i think this is because .tar is intended to create portable copies,...

Q: am i on the right track here or all wet ? i plan to use .tar.gz and .tar archives to move my data from MINT15 into LMDE MINT 201402 sometime here in 1Q2014 ...

if i remember rightly from my move from Ubuntu 12.04LTS to MINT 15 I used that method successfully but-- that time I use an NTFS format shared drive ( which I'm going to remove from the box ) and the new one will be Linux format. i don't care if it shares with windows, or not.
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Linux Mint is funded by ads and donations.
 

Re: migrating files using tar.gz

Postby GeneBenson on Thu Feb 20, 2014 1:13 am

Hi mike acker,

In a terminal run: man tar
This will tell you everything you need to know. The relevant part when extracting is:

Code: Select all
--no-same-owner
           extract files as yourself (default for ordinary users)

--same-owner
           try extracting files with the same ownership as exists in the archive (default for superuser)

Hope this helps. :D
GeneBenson
Level 3
Level 3
 
Posts: 171
Joined: Fri Sep 17, 2010 9:55 pm

Re: migrating files using tar.gz

Postby mike acker on Thu Feb 20, 2014 11:37 am

GeneBenson wrote:Hi mike acker,

In a terminal run: man tar
This will tell you everything you need to know. The relevant part when extracting is:

Code: Select all
--no-same-owner
           extract files as yourself (default for ordinary users)

--same-owner
           try extracting files with the same ownership as exists in the archive (default for superuser)

Hope this helps. :D


ah-HA!!! i figure then it's the case that when i use the GUI it applies the --no-same-owner attribute. I just installed a new 1TB drive in my main box to receive the backups from my MINT 15 system so these can move to my new LMDE system, hopefully arriving pretty soon
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: migrating files using tar.gz

Postby viking777 on Thu Feb 20, 2014 12:14 pm

i plan to use .tar.gz and .tar archives to move my data from MINT15 into LMDE MINT 201402 sometime here in 1Q2014 ...


That is what Clem wrote Mint Backup for - he'll be really offended if you don't use it :wink:

http://community.linuxmint.com/tutorial/view/2

See section D.
Fujitsu Lifebook AH532. Intel i5 processor, 6Gb ram, Intel HD3000 graphics, Intel Audio/wifi. Realtek RTL8111/8168B Ethernet.Lubuntu 13.10,Ubuntu12.10 (Unity), Mint16 (Cinnamon), Manjaro (Xfce).
Image
User avatar
viking777
Level 14
Level 14
 
Posts: 5153
Joined: Mon Dec 01, 2008 11:21 am

Re: migrating files using tar.gz

Postby mike acker on Thu Feb 20, 2014 1:47 pm

viking777 wrote:
i plan to use .tar.gz and .tar archives to move my data from MINT15 into LMDE MINT 201402 sometime here in 1Q2014 ...


That is what Clem wrote Mint Backup for - he'll be really offended if you don't use it :wink:

http://community.linuxmint.com/tutorial/view/2

See section D.


ok wow!! I'll be learning how to use this critter!!

tar is "OK" but when you backup a library like /Music you end up with /Music as the top level directory in the archive and then when you restore it on a new system you end up with /Music/Music ...
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: migrating files using tar.gz

Postby viking777 on Fri Feb 21, 2014 7:06 am

and then when you restore it on a new system you end up with /Music/Music ...


Oh yes, I have been there too! But there is a cunning and quick command line way out of that hole. I goes like this:

Code: Select all
mv Music/Music Music2


This renames the second or 'inner' Music folder to Music2 and places it in your /home/username folder (which is where your terminal works by default).

Code: Select all
rmdir Music


This removes the first or 'outer' music folder which is now empty.

Code: Select all
mv Music2 Music


This renames the old inner Music folder (the one with all your music in it) back to Music.

The whole thing only takes seconds, I show it above in three steps in order to explain what is happening, but it can all be written as one command ie>

Code: Select all
mv Music/Music Music2 && rmdir Music && mv Music2 Music


Create a couple of 'test' folders one inside the other and create a test file inside the inner one then run the command above substituting the word 'test' for 'Music' and you will see how easy it is to do without risking your Music files (although there is no risk really, it is just command line magic).
Fujitsu Lifebook AH532. Intel i5 processor, 6Gb ram, Intel HD3000 graphics, Intel Audio/wifi. Realtek RTL8111/8168B Ethernet.Lubuntu 13.10,Ubuntu12.10 (Unity), Mint16 (Cinnamon), Manjaro (Xfce).
Image
User avatar
viking777
Level 14
Level 14
 
Posts: 5153
Joined: Mon Dec 01, 2008 11:21 am

Re: migrating files using tar.gz

Postby mike acker on Fri Feb 21, 2014 9:33 am

viking777 wrote:
and then when you restore it on a new system you end up with /Music/Music ...


Oh yes, I have been there too! But there is a cunning and quick command line way out of that hole. I goes like this:{snip}

Create a couple of 'test' folders one inside the other and create a test file inside the inner one then run the command above substituting the word 'test' for 'Music' and you will see how easy it is to do without risking your Music files (although there is no risk really, it is just command line magic).


that's an interesting approach.... i thought about it and didn't try it because certain of these files.... /Documents , /Music , /Pictures , /Videos ... --- are the "user libraries." I was afraid if i deleted one of these I could not properly put it back. the old PKZIP program had option switches for dealing with this; I'll have to read the man tar stuff some more and see if i can figure it out. I need to learn more about scripting. command line provides access to all the nifty switches.... but the commands are frustrating to edit,-- one wrong character and ZORK!. if i can edit these with BlueFish and save them it will be better.

Migrating a system really isn't too bad though after the major libraries are done. the tricky one is Thunderbird,-- I have to go get the data out of that default folder, and Firefox does a lot of selected(non critical) automatic logons for me -- i have to get those, too
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: migrating files using tar.gz

Postby viking777 on Fri Feb 21, 2014 10:31 am

When I reinstall or install new distros I just copy the entire /home/username/.mozilla and /home/username/.thunderbird folders from the old to the new distro. It has always worked for me. The trick is to copy them over before you run either Firefox or Thunderbird on the new install, that way they will use the copied profile folders instead of creating new ones at first run.
Fujitsu Lifebook AH532. Intel i5 processor, 6Gb ram, Intel HD3000 graphics, Intel Audio/wifi. Realtek RTL8111/8168B Ethernet.Lubuntu 13.10,Ubuntu12.10 (Unity), Mint16 (Cinnamon), Manjaro (Xfce).
Image
User avatar
viking777
Level 14
Level 14
 
Posts: 5153
Joined: Mon Dec 01, 2008 11:21 am

Re: migrating files using tar.gz

Postby mike acker on Fri Feb 21, 2014 11:37 am

viking777 wrote:When I reinstall or install new distros I just copy the entire /home/username/.mozilla and /home/username/.thunderbird folders from the old to the new distro. It has always worked for me. The trick is to copy them over before you run either Firefox or Thunderbird on the new install, that way they will use the copied profile folders instead of creating new ones at first run.


way cool. I'll definitely try that this time!!

thanks
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: migrating files using tar.gz

Postby mike acker on Sat Feb 22, 2014 7:51 pm

viking777 wrote:When I reinstall or install new distros I just copy the entire /home/username/.mozilla and /home/username/.thunderbird folders from the old to the new distro. It has always worked for me. The trick is to copy them over before you run either Firefox or Thunderbird on the new install, that way they will use the copied profile folders instead of creating new ones at first run.


I had a great deal of trouble with this. first off, the directory
Code: Select all
~/.mozilla

isn't accessible through the GUI tools.

Second: in moving data of this type from one O/S to another the chown settings have to be changed to the new owner. otherwise the data isn't usable in the new system. Fortunately tar helps with this. on the sending system I did this:
Code: Select all
cd ~/.mozilla
 ~/.mozilla $ tar -cf ~/Public/Mozilla.tar *


this created an archive named Mozilla.tar in the sending system, ~/Public directory. This I then copied this to the backup drive using Nemo.

next, I got the MINT ISO 201402 downloaded and installed. I had a hard time getting the new disk formatted and partitioned properly. it probably would have run normally except I think my first burned DVD crapped out on me and I had to burn a new DVD. anyway, I thought I did because the install hanged half way through.

so, I thought I'd have to reformat the drive and I did that and after that I couldn't do anything with the install ISO. I had to install the MINT 15 ISO which then formatted the disk for me and then go back to the LMDE 201402 ISO and the second time it made it, after i figured out i had to double click on MOUNT in order to get the program to accept the mount at / which I'm not sure how i figured out it had to mount at / but i think it told me i didn't have anything mounted at / so i kinda guessed....

once i got the new system up i had to get the data from the backup drive. first i used archive-manager to extract the mozilla.tar to ~/Documents/mozilla-transfer
then i had to copy the firefox directory. that had to be removed first and because i was working in the ~/.mozilla directory i had to use command line. i finally found the command
Code: Select all
cd ~/.mozilla
rm -rf firefox

after that i was able to copy the directory in
Code: Select all
cp -r ~/Documents/mozilla-transfer/firefox ~/.mozilla


hopefully they could hear me cussing all the way to mozilla HQ: where is the export/import user options switch ?????

anyway, all's well that ends well for Firefox anyway: when I launched it after the option copy all my history, bookmarks, and add-ons were preserved. probably passwords too: it let me on this board automagically

anyway, that's my summary of the First Inning.
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: migrating files using tar.gz

Postby mike acker on Sat Feb 22, 2014 8:42 pm

the same technique worked with ~/.thunderbird , the key file being the <random string>.default file

i copied that in ok,-- again I had to use command line cp command in order to access the .thunderbird area
and i had to edit the profiles.ini file to change the parameter setting the name of the default file and
i had to change use last profile to 0 for 1 run

this does not bring pgp keys
i'll have to re-ipl the old system and get the keys exported.
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: migrating files using tar.gz STOP

Postby mike acker on Sat Feb 22, 2014 9:00 pm

STOP

i am getting this error trying to extract the backup archives.

http://napfn.com/errorsetting.png

i have years worth of research that needs to come over and I can't deal with this stop.
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: migrating files using tar.gz

Postby viking777 on Sun Feb 23, 2014 7:25 am

I am utterly baffled by what you have said there. For me, copying these files over consists of 'right click copy', 'right click paste' - it is as difficult as that. Command line ?? tar ?? chown?? you don't need any of that. I have 4 different distros with 4 different usernames and they all read each others files without difficulty. Of course they do have to have the same uid as each other, but that is normally taken care of by default as must Ubuntu/Mint uid's for the first user is 1000. I am fairly sure LMDE does the same. As for your assertion that .mozilla is not accessible in your file manager, I am gob smacked - you must have a seriously duff file manager is all I can say. The worst file manager on the planet is Nautilus and even it can manage that task.

You know that anything with a dot in front of it is a 'hidden' file? - to see them you just hit Ctl/h in most file managers.

Another suggestion, - forgive me if I have misinterpreted something, it happens often - are you trying to replace one distro with another? If so why? This isn't windows, anyone who has less than two linux distros installed at the same time is a fool to themselves imho. If you want to replace one distro with another install them side by side to begin with then, when you have copied over all the files and folders that you want from one to the other, delete the first distro. Better still keep it until you want to install another distro, that way if one breaks you just boot straight into the other.

Dual/multi booting is so simple these days, just click the 'install alongside' or 'something else' button on the installer when you want to switch distros (whichever you see - installers vary) and choose how much disk space you want to give to each one.

xenopeek has a tutorial on multi booting here using the 'something else' method: viewtopic.php?f=42&t=122276 have a look at it if you are interested, but don't let me twist your arm about anything, and sorry if my suggestion has led you a merry dance, it wasn't intentional.
Fujitsu Lifebook AH532. Intel i5 processor, 6Gb ram, Intel HD3000 graphics, Intel Audio/wifi. Realtek RTL8111/8168B Ethernet.Lubuntu 13.10,Ubuntu12.10 (Unity), Mint16 (Cinnamon), Manjaro (Xfce).
Image
User avatar
viking777
Level 14
Level 14
 
Posts: 5153
Joined: Mon Dec 01, 2008 11:21 am

Re: migrating files using tar.gz

Postby killer de bug on Sun Feb 23, 2014 7:31 am

viking777 wrote:You know that anything with a dot in front of it is a 'hidden' file? - to see them you just hit Ctl/h in most file managers.


No he don't. That's obvious... :shock:

viking777 wrote: anyone who has less than two linux distros installed at the same time is a fool to themselves imho.

Come on. I have only LMDE. No other OS :lol:
Ok I have some OS installed in VB. But on my computer, I have only LMDE :lol:
If I have seen further it is by standing on the shoulders of giants. [Isaac Newton]
User avatar
killer de bug
Level 7
Level 7
 
Posts: 1544
Joined: Tue Jul 08, 2008 1:49 pm
Location: Austria

Re: migrating files using tar.gz

Postby mike acker on Sun Feb 23, 2014 8:19 am

sorry, guys, I'm having a terrible time, but I ain't giving up and i ain't going back to that Commie system that comes from the land of the Red*

the reason I try to migrate stuff with tar is that when I don't do that I have permissions issues on the receiving system as the file ends up with the wrong owner. when I switched from Ubuntu 12.04 LTS to Cinamon 14 - I had a terrible time with this. I have 500G of data to convert.

^h -- reveals the hidden files ( thanks killer -- I didn't know that ) -- using NEMO -- but not in the Archive Manager. after I learned the commands in the Firefight I mean the Firefox conversion -- I managed the Thunderbird <--->.default folder rather handilly. I think that StartWithLastProfile switch threw me earlier -- this time I set it to 0 for the first run and Thunderbird accepted the earlier profile. I have set it back to 1 now with no apparent ill affect-- I suspect it should read "ReloadProfileData=y" or "ImportProfileData=y" as I suspect that's what's really happening with that.

note-- I had tried Clem's backup utility -- but it hanged on me, part way through. the data archives that I have go back to 1992 and contain many file formats -- saved web pages, there's old .wri files out there-- stuff collected from DOS, Windows 3.11, XP, Win7, Ubuntu 12.04, and Cinnamon 15.
~~~

anyway, after coffee this morning I'm going to reboot my new LMDE 201402 and create a new backup of the /Documents directory off the Cinnamon 15 (current) system onto the backup drive and then try to pull that into the LMDE. There could be some difference in the version of the tar that is in use. Also I may try this from command line using the --ignore-command-error switch .

I remember fighting this battle back in the days of MVS/370 on the backup-restore software. you might have a reguest to "restore file "xyz" -- and "xyz" is someplace in a 5 tape backup file. you start the restore and get a tape error on the 3d tape and the program cancels without restoring "xyz" -- even though the backup of "xyz" didn't cause the error. it's important to be able to bypass these errors on these backup/restore runs ( but you'll need a log dataset -- and that should display at the end of the process -- or at least a dialog should ask you if you want to view it and tell you that you have it and where to find it )

oh well, after coffee I'll try another run during this morning's Blue Grass Festival on WMUK
~~~
sda1=lmde 201402 ; sdb1=Backup_Vol2 ; sdc1=Mint 15, u-base. I can't put any more in right now because my power supply (450w) has 4 SATA power connectors.
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: migrating files using tar.gz

Postby mike acker on Sun Feb 23, 2014 11:20 am

This morning I re-extracted the documents.tar.gz archive but using the command line interface rather than the GUI "Archive Manager"
Code: Select all
tar -xvf /media/mike/Backup_Vol2/Backups-Feb-2014/Documents.tar.gz -C ~/Documents --ignore-command-error >listing.txt


the first thing I noticed: I ran MUCH quicker.

browsing through the resulting report I noticed some error messages:
e.g.
Code: Select all
workarea1/jenvideo1/workarea4/slide5.jpg: Mode differs


following that I decided to run the verify test-- they call it a "difference test" --

Code: Select all
tar -df /media/mike/Backup_Vol2/Backups-Feb-2014/Documents.tar.gz -C ~/Documents >verify_error_list.txt


running a grep command to see if any other error was reported:
Code: Select all
grep -v -e "Mode differs" verify_error_list.txt

found nothing

checking one of the files reported with a "Mode Differs" error did not have a problem; file worked OK. I'll check a few more after _Grassroots_ and I want to run a verify from the source system to the backup archive. this seems to be an option the GUI archive manager doesn't offer but in migrating large directories it seems essential...

Q: What does the error mean,.... and why am I getting it ?
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: app programs

Postby mike acker on Sun Feb 23, 2014 4:31 pm

so far, so good: the LMDE 201402 RC boots and runs like a dream. Note the basics of my system box in my signature line.

I'm just dealing with what are most likely individual problems. I'll continue to post my notes here on my switchover unless the admins ask me to do something different.

apps.

1. LibreOffice
201402 has version 4.1 installed; I want to upgrade to 4.2

earlier, I used this process:
http://www.libre-software.net/how-to-install-libreoffice-on-ubuntu-linux-mint

I Performed this on LMDE/201402 and it appears to have worked perfect. Remember to select LibreOffice 4.2 base and add that to "panel" and then edit the icon. They have ICONS for me this time; on MINT15 I had to create my own ( just a png graphic 32x32 pxels ) .

Wireless HP Officejet 6500 found and configured.

2. Simplescan
found and added simplescan to panel. tested. works fine.
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Re: migrating files using tar.gz

Postby killer de bug on Sun Feb 23, 2014 5:12 pm

We are very far from your initial question...
And please remember that this forum is not your personal blog :wink:
If I have seen further it is by standing on the shoulders of giants. [Isaac Newton]
User avatar
killer de bug
Level 7
Level 7
 
Posts: 1544
Joined: Tue Jul 08, 2008 1:49 pm
Location: Austria

Re: migrating files using tar.gz

Postby GeneBenson on Mon Feb 24, 2014 3:15 am

Hi mike acker,

Glad to hear it all worked out in the end. Although you probably learned a lot from this exercise there are, as viking777 pointed out, much easier ways of doing what you wanted. Just a few tips:

In your first post you mentioned a new 1TB HDD. If you haven't already done so creating an extra "data" partition for sharing data between different OS's is a good idea.

You can see hidden files in an Archive Manager. At least in Engrampa (MATE) you can. Open an archive, choose a location to extract to, then in the right hand column right click. You should (might) see an option to "Show hidden files".

Be careful when installing apps meant for Ubuntu. Although LibreOffice is probably an exception, there are any number of posts here complaining that installing a Ubuntu app wreaked their system. Saying that an app is also for Linux Mint is no assurance because they are probably referring to the Main Edition of LM which is based on Ubuntu. Unless it specifically says it is also compatible with Debian use caution.

P.s. I started working on MVS/370 in 1984 and your story of bad tapes brought back all sorts of nightmares!! :shock: :?
GeneBenson
Level 3
Level 3
 
Posts: 171
Joined: Fri Sep 17, 2010 9:55 pm

Re: migrating files using tar.gz

Postby mike acker on Mon Feb 24, 2014 11:51 am

killer de bug wrote:We are very far from your initial question...
And please remember that this forum is not your personal blog :wink:


i apologize for mis-using the forum.
could the administrator move this thread to the chat area? would that be OK?
Home assembled box using ASUS M5A88-M motherboard and AMD Phenom II X4 3.4GHz cpu
User avatar
mike acker
Level 4
Level 4
 
Posts: 340
Joined: Wed Jul 31, 2013 6:29 pm
Location: Kalamazoo, MI

Linux Mint is funded by ads and donations.
 
Next

Return to Chat about Linux Mint

Who is online

Users browsing this forum: No registered users and 6 guests