I'd like to provide a mirror, but rsync is too slow

deb http://debian.linuxmint.com

I'd like to provide a mirror, but rsync is too slow

Postby killerhorse on Fri May 10, 2013 4:55 am

Hello,

I would like to create an apt-mirror for the LMDE repository.
The problem is, that the debian.linuxmint.com server is incredible slow.
I'running rsync for days and it would take !months! to complete the synchronisation of the hole 600 GB.
Is there any other server I can use for the initial download/synchronisation?

Christian
killerhorse
Level 1
Level 1
 
Posts: 4
Joined: Wed May 08, 2013 4:41 pm

Linux Mint is funded by ads and donations.
 

Re: I'd like to provide a mirror, but rsync is too slow

Postby Fesiitis on Fri May 10, 2013 7:07 am

Hello, Christian!

Just open Terminal and run the following command: mint-choose-debian-mirror, then you can choose another mirror, which is better for you. ;)
Fesiitis
Level 2
Level 2
 
Posts: 53
Joined: Sun Feb 19, 2012 8:02 am

Re: I'd like to provide a mirror, but rsync is too slow

Postby killerhorse on Fri May 10, 2013 7:55 am

This will not work, because my server doesn't run linuxmint.
I don't want to use a mirror, i'd like to provide a mirror. So I don't need an programm who changes the entries in my /etc/apt/sources.list to install packages using apt-get or aptitude, I need a source to sync with rsync with.

Christian
killerhorse
Level 1
Level 1
 
Posts: 4
Joined: Wed May 08, 2013 4:41 pm

Re: I'd like to provide a mirror, but rsync is too slow

Postby i.r.id10t on Thu May 16, 2013 10:08 pm

I'm in a similar boat - I already mirror Debian and Ubuntu, and while I use Mint, I also use Mint as a desktop for the Linux class I teach. I'd like to set up a Mint mirror, but I don't want to duplicate data/packages that I already have on the sever via Debian and Ubuntu.

viewtopic.php?f=190&t=133153

Ideas?
i.r.id10t
Level 1
Level 1
 
Posts: 11
Joined: Mon Jun 25, 2012 9:26 am

AW: I'd like to provide a mirror, but rsync is too slow

Postby killerhorse on Fri May 17, 2013 6:28 pm

Currently I'm synchronizing with one of the rsync mirrors. My mirror is online. I sent a mail one week ago and another one some days ago to root at linuxmint dot com. I tried to reach someone in the irc chat and here in the forum. But it seems that there are no further mirrors needed, because nobody of the Linux mint team sends any answer...
killerhorse
Level 1
Level 1
 
Posts: 4
Joined: Wed May 08, 2013 4:41 pm

Re: I'd like to provide a mirror, but rsync is too slow

Postby catweazel on Fri May 17, 2013 7:43 pm

http://apt-mirror.github.io/
http://www.howtoforge.com/local_debian_ubuntu_mirror
I found those two links with a simple google search. You might try it sometime.
Mint Testing Team & Mint Donor #3606
KDE 4.12.0, custom preemptive kernel 3.12.5,
Intel i7 4770K @ 4.7GHz, 16GB 2666MHz XMP,
4 Samsung 840 PRO 512GB SSDs in RAID0,
6TB HW RAID10, dual 24" Acer X243H,
Gigabyte nVidia GTX 680 Super Overclock
User avatar
catweazel
Level 7
Level 7
 
Posts: 1656
Joined: Fri Oct 12, 2012 9:44 pm

Re: I'd like to provide a mirror, but rsync is too slow

Postby killerhorse on Fri May 17, 2013 9:15 pm

Sorry, I know... my English is not the best. It seems, that no one understands me.
I don't asked for help how I can create a mirror.
I don't like to create a lokal mirror.
I already have a working mirror. I will not synchronize it with apt-mirror, because this page:
http://www.linuxmint.com/mirrors.php
says that an official mirror should be synchronized with rsync.
My mirror only should be added to the mirror list on this Page: http://www.linuxmint.com/mirrors.php and to any other lists of mirrors by Linux Mint.
The problem is, that it's not possible to get any response from the Linux Mint team. And because of that I think there is no need for aditional Mirrors.
Another Problem is, that the Main Server is so slow, that it's not possible to keep the mirror up to date. I can use another mirror to synchronize with, but they seem to have the same problem, because they have different datasets, so some of them are also not up to date.
killerhorse
Level 1
Level 1
 
Posts: 4
Joined: Wed May 08, 2013 4:41 pm

Re: I'd like to provide a mirror, but rsync is too slow

Postby catweazel on Fri May 17, 2013 11:34 pm

rsync is not the problem. It is only as fast as the mirror you are pulling from. I use rsync to shuffle hundreds of gigabytes between SSDs and 10,000RPM WD VelociRaptors in RAID 10; it simply screams through the work.

Choose a faster source mirror and only pull critical or security updates directly from the main repository. Alternatively pull everything from a faster mirror then only update from the Mint repo.
Mint Testing Team & Mint Donor #3606
KDE 4.12.0, custom preemptive kernel 3.12.5,
Intel i7 4770K @ 4.7GHz, 16GB 2666MHz XMP,
4 Samsung 840 PRO 512GB SSDs in RAID0,
6TB HW RAID10, dual 24" Acer X243H,
Gigabyte nVidia GTX 680 Super Overclock
User avatar
catweazel
Level 7
Level 7
 
Posts: 1656
Joined: Fri Oct 12, 2012 9:44 pm

Re: I'd like to provide a mirror, but rsync is too slow

Postby ketoth on Tue Jul 23, 2013 3:37 am

Maybe you found a solution meanwhile, if not:

Code: Select all
rsync debian.lth.se::lmde


Doesn't really matter where you are located, this Swedish mirror should be way faster than the Mint team's main server.
Here, an excerpt from Clem's email to mirror maintainers just a month ago, other options to try:
We've come to the point where our single server cannot serve all the rsync connections. So instead of all mirrors syncing directly against it, we recommend some of the mirrors sync with other mirrors instead of syncing with us directly.

Below are the mirrors which provide rsync connections themselves. We'd like these mirrors to continue to sync directly with us:

Japan - JAIST - ftp.jaist.ac.jp::pub/Linux/linuxmint/debian/
France - Gwendal Le Bihan - mint-mirror.gwendallebihan.net::lmde-packages
Sweden - Lund University - debian.lth.se::lmde
UK - University of Kent UK Mirror Service - rsync.mirrorservice.org::debian.linuxmint.com
Australia - Internode - mirror.internode.on.net::linuxmint-debian
Proud Linux Mint Debian user since 04/2011
mirror.rts-informatique.fr mirror tech maintainer

Likes GPL and Creative Commons and people using them
Somewhere between Linus Torvalds and Richard Stallman
ketoth
Level 2
Level 2
 
Posts: 81
Joined: Sat Dec 31, 2011 8:01 pm

Linux Mint is funded by ads and donations.
 

Return to Update Packs Repositories

Who is online

Users browsing this forum: No registered users and 1 guest