I'd like to provide a mirror, but rsync is too slow

Archived topics about LMDE 1 and LMDE 2
Locked
killerhorse

I'd like to provide a mirror, but rsync is too slow

Post by killerhorse »

Hello,

I would like to create an apt-mirror for the LMDE repository.
The problem is, that the debian.linuxmint.com server is incredible slow.
I'running rsync for days and it would take !months! to complete the synchronisation of the hole 600 GB.
Is there any other server I can use for the initial download/synchronisation?

Christian
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
killerhorse

Re: I'd like to provide a mirror, but rsync is too slow

Post by killerhorse »

This will not work, because my server doesn't run linuxmint.
I don't want to use a mirror, i'd like to provide a mirror. So I don't need an programm who changes the entries in my /etc/apt/sources.list to install packages using apt-get or aptitude, I need a source to sync with rsync with.

Christian
i.r.id10t

Re: I'd like to provide a mirror, but rsync is too slow

Post by i.r.id10t »

I'm in a similar boat - I already mirror Debian and Ubuntu, and while I use Mint, I also use Mint as a desktop for the Linux class I teach. I'd like to set up a Mint mirror, but I don't want to duplicate data/packages that I already have on the sever via Debian and Ubuntu.

http://forums.linuxmint.com/viewtopic.p ... 0&t=133153

Ideas?
killerhorse

AW: I'd like to provide a mirror, but rsync is too slow

Post by killerhorse »

Currently I'm synchronizing with one of the rsync mirrors. My mirror is online. I sent a mail one week ago and another one some days ago to root at linuxmint dot com. I tried to reach someone in the irc chat and here in the forum. But it seems that there are no further mirrors needed, because nobody of the Linux mint team sends any answer...
User avatar
catweazel
Level 19
Level 19
Posts: 9763
Joined: Fri Oct 12, 2012 9:44 pm
Location: Australian Antarctic Territory

Re: I'd like to provide a mirror, but rsync is too slow

Post by catweazel »

http://apt-mirror.github.io/
http://www.howtoforge.com/local_debian_ubuntu_mirror
I found those two links with a simple google search. You might try it sometime.
"There is, ultimately, only one truth -- cogito, ergo sum -- everything else is an assumption." - Me, my swansong.
killerhorse

Re: I'd like to provide a mirror, but rsync is too slow

Post by killerhorse »

Sorry, I know... my English is not the best. It seems, that no one understands me.
I don't asked for help how I can create a mirror.
I don't like to create a lokal mirror.
I already have a working mirror. I will not synchronize it with apt-mirror, because this page:
http://www.linuxmint.com/mirrors.php
says that an official mirror should be synchronized with rsync.
My mirror only should be added to the mirror list on this Page: http://www.linuxmint.com/mirrors.php and to any other lists of mirrors by Linux Mint.
The problem is, that it's not possible to get any response from the Linux Mint team. And because of that I think there is no need for aditional Mirrors.
Another Problem is, that the Main Server is so slow, that it's not possible to keep the mirror up to date. I can use another mirror to synchronize with, but they seem to have the same problem, because they have different datasets, so some of them are also not up to date.
User avatar
catweazel
Level 19
Level 19
Posts: 9763
Joined: Fri Oct 12, 2012 9:44 pm
Location: Australian Antarctic Territory

Re: I'd like to provide a mirror, but rsync is too slow

Post by catweazel »

rsync is not the problem. It is only as fast as the mirror you are pulling from. I use rsync to shuffle hundreds of gigabytes between SSDs and 10,000RPM WD VelociRaptors in RAID 10; it simply screams through the work.

Choose a faster source mirror and only pull critical or security updates directly from the main repository. Alternatively pull everything from a faster mirror then only update from the Mint repo.
"There is, ultimately, only one truth -- cogito, ergo sum -- everything else is an assumption." - Me, my swansong.
ketoth

Re: I'd like to provide a mirror, but rsync is too slow

Post by ketoth »

Maybe you found a solution meanwhile, if not:

Code: Select all

rsync debian.lth.se::lmde
Doesn't really matter where you are located, this Swedish mirror should be way faster than the Mint team's main server.
Here, an excerpt from Clem's email to mirror maintainers just a month ago, other options to try:
We've come to the point where our single server cannot serve all the rsync connections. So instead of all mirrors syncing directly against it, we recommend some of the mirrors sync with other mirrors instead of syncing with us directly.

Below are the mirrors which provide rsync connections themselves. We'd like these mirrors to continue to sync directly with us:

Japan - JAIST - ftp.jaist.ac.jp::pub/Linux/linuxmint/debian/
France - Gwendal Le Bihan - mint-mirror.gwendallebihan.net::lmde-packages
Sweden - Lund University - debian.lth.se::lmde
UK - University of Kent UK Mirror Service - rsync.mirrorservice.org::debian.linuxmint.com
Australia - Internode - mirror.internode.on.net::linuxmint-debian
Locked

Return to “LMDE Archive”