[solved] rsyncing to my NAS

[AndyMH]

It's a synology DS216J running DSM 7.2.

It works, ran this as a test (second time so nothing new):

Code: Select all

andy@M720 ~ $ rsync -avzx --stats --delete --recursive --info=progress2 /home/andy/Downloads andy@diskstation.local:/volume1/homes/backup
andy@diskstation.local's password: 
sending incremental file list
Downloads/
              0   0%    0.00kB/s    0:00:00 (xfr#0, to-chk=0/506)  

Number of files: 506 (reg: 478, dir: 28)
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 0
Total file size: 37,569,989,812 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 14,565
Total bytes received: 54

sent 14,565 bytes  received 54 bytes  2,249.08 bytes/sec
total size is 37,569,989,812  speedup is 2,569,942.53

But...
It prompts for a password, my intention is to set this up as a script for backup running automatically as a cron job. Is there any way to pass the password in the command?

I did try (with the real pwd), but no joy:

Code: Select all

andy@M720 ~ $ rsync -avzx --stats --delete --recursive --info=progress2 /home/andy/Downloads andy:password@diskstation.local:/volume1/homes/backup
ssh: Could not resolve hostname andy: Name or service not known

If the answer is no, then I'll have to look at an nfs share, but I read somewhere ssh is faster.

[rene]

Rather than automate sending a password on the computer you should set up public key authentication for user "andy" on the NAS via e.g. https://samuelsson.dev/log-in-with-ssh- ... ology-nas/

[AndyMH]

rene, thanks for that. Looks like exactly what I need. I'll have a detailed look tomorrow.

[rene]

Rather than doing things manually on the NAS as per that link you can by the way also more easily use ssh-copy-id. That is, things should work after:

1. If not already set "Enable user home service" on the NAS as per the link.

2. Test if normal password-based SSH works with from a computer ssh andy@diskstation.local.

Note that if you're also "andy" on the computer you can simply say ssh diskstation.local without the andy@ part, and that if you are not "andy" on the computer you can on the computer create/edit ~/.ssh/config to be/have

Code: Select all

Host diskstation.local
	User	andy

after which you can again leave out the andy@.

3. On the computer generate a keypair for general LAN SSH if you don't yet have a pair to use for the purpose:

Code: Select all

ssh-keygen -t rsa -b 4096 -N ""

This will generate a pair ~/.ssh/id_rsa{,.pub}. If you want you can also generate a specific pair with e.g.

Code: Select all

ssh-keygen -t rsa -b 4096 -N "" -f ~/.ssh/diskstation

but I advise former.

4. On the computer copy over the public key to the NAS (can still leave out andy@ if you are "andy" or if you have set up ~/.ssh/config as per 2):

Code: Select all

ssh-copy-id andy@diskstation.local

or

Code: Select all

ssh-copy-id -i ~/.ssh/diskstation.pub andy@diskstation.local

This should prompt you for andy's password on the NAS and then install the public key in the NAS's ~andy/.ssh/.

5. Test if you can now login without password with from the computer (can still [etc.]):

Code: Select all

ssh andy@diskstation.local

or

Code: Select all

ssh -i ~/.ssh/diskstation andy@diskstation.local

6. If you elected for the specific diskstation{,.pub} keypair, on the computer edit/create ~/.ssh/config with

Code: Select all

Host diskstation.local
	IdentityFile	~/.ssh/diskstation

or then if you already created/edited as per 2,

Code: Select all

Host diskstation.local
	User		andy
	IdentityFile	~/.ssh/diskstation

7. Test if now simple, key-based SSH works via no more than ssh diskstation.local. If so it will also work to say e.g.

Code: Select all

rsync -avzx --stats --delete --recursive --info=progress2 /home/andy/Downloads diskstation.local:/volume1/homes/backup

8. If you used the general LAN SSH key pair feel free to also ssh-copy-id [user@]foo.local for another LAN-system foo.local to also on foo.local have setup key-based authentication.

[AndyMH]

Thanks for taking the time to provide that guidance. Yes, the user is andy on both the local mint pc and on the NAS (ids are different, 1000 locally, think it is 1028 on the NAS - trust synology to do it differently!).

I be testing one step at a time, rsync already works as indicated in my first post.

Synology is flagging up as a security risk that I have not changed the default ssh port (22). Any advice on port ranges to either use or avoid?

[rene]

Synology is flagging up as a security risk that I have not changed the default ssh port (22). Any advice on port ranges to either use or avoid?

No, although I do have advise on secuwity crap to avoid...

I.e., it's on a private LAN which your NAS is supposedly on really not useful and frankly not even if the NAS is/were reachable from the internet, since locating your SSH service on whichever port is exactly one port scan away. For a publicly reachable NAS I'd advise instead to disable password-based SSH after setting up key-based as per above, and on a not publicly reachable one I'd advise to do same if you insist, but fully ignore said "security risk" as the better solution.

Now, if it's the only way to shut up the dumb thing and you decide to use a different port anyway, note that you can also have that port in ~/.ssh/config as e.g.

Code: Select all

Host diskstation.local
	Port		1234

so that you won't have to specify it.

By the way, also that you can use e.g.

Code: Select all

Host nas
	HostName	diskstation.local
	[ ... ]

to be able to say ssh nas rather than ssh diskstation.local if you'd like that better...

[AndyMH]

It's only on the LAN, not visible to the outside world.

[AndyMH]

It worked but still requires a password.

Enable user home service

Checked - enabled.

Test if normal password-based SSH works with from a computer ssh

Yes:

Code: Select all

andy@M720 ~ $ ssh andy@diskstation.local
andy@diskstation.local's password: 

Using terminal commands to modify system configs, execute external binary
files, add files, or install unauthorized third-party apps may lead to system
damages or unexpected behavior, or cause data loss. Make sure you are aware of
the consequences of each command and proceed at your own risk.

Warning: Data should only be stored in shared folders. Data stored elsewhere
may be deleted when the system is updated/restarted.

andy@DiskStation:~$ 

Note - did try ssh diskstation.local and it works (after logging out of the diskstation) as user andy on both.

On the computer generate a keypair for general LAN SSH if you don't yet have a pair to use for the purpose:

Code: Select all

andy@M720 ~ $ ssh-keygen -t rsa -b 4096 -N ""
Generating public/private rsa key pair.
Enter file in which to save the key (/home/andy/.ssh/id_rsa): 
Your identification has been saved in /home/andy/.ssh/id_rsa
Your public key has been saved in /home/andy/.ssh/id_rsa.pub
The key fingerprint is:
SHA256: <redacted>
The key's randomart image is:
+---[RSA 4096]----+
|    changed below at random for this post  |
|        oo +o. o.|
|       .o +.. = +|
|    |
|     o .S =.oo  +|
|   |
|        ooo o+ = |
|         ..=..o  |
|          o..+o  |
+----[SHA256]-----+

On the computer copy over the public key to the NAS (can still leave out andy@ if you are "andy" or if you have set up ~/.ssh/config as per 2):

Code: Select all

andy@M720 ~ $ ssh-copy-id andy@diskstation.local
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
andy@diskstation.local's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'andy@diskstation.local'"
and check to make sure that only the key(s) you wanted were added.

Test if you can now login without password with from the computer (can still [etc.]):

Code: Select all

andy@M720 ~ $ ssh andy@diskstation.local
andy@diskstation.local's password: 

Using terminal commands to modify system configs, execute external binary
files, add files, or install unauthorized third-party apps may lead to system
damages or unexpected behavior, or cause data loss. Make sure you are aware of
the consequences of each command and proceed at your own risk.

Warning: Data should only be stored in shared folders. Data stored elsewhere
may be deleted when the system is updated/restarted.

andy@DiskStation:~$ 

It asked for a password "and check to make sure that only the key(s) you wanted were added" - how would I do this?

I have not rebooted the synology?

[AndyMH]

EDIT - had a look around for where the keys might be, in /etc/ssh on the NAS:

Code: Select all

andy@DiskStation:/etc/ssh$ ls
sshd_config           ssh_host_ecdsa_key.pub    ssh_host_key.pub
ssh_host_dsa_key      ssh_host_ed25519_key      ssh_host_rsa_key
ssh_host_dsa_key.pub  ssh_host_ed25519_key.pub  ssh_host_rsa_key.pub
ssh_host_ecdsa_key    ssh_host_key

ssh_host_rsa_key.pub looked promising, ran cat on it, then had a look locally on my mint pc in:

Content of ssh_host_rsa_key.pub on the NAS is identical to the content of id_rsa.pub locally. No idea if this helps, well beyond my knowledge and skill set.

[rene]

I have not rebooted the synology?

On a "normal" computer that wouldn't be necessary and that earlier https://samuelsson.dev/log-in-with-ssh- ... ology-nas/ link seems to indicate it wouldn't be on Synology either -- but doesn't hurt I guess.

However, first verify that all is as expected. I.e., log in simply as ssh diskstation.local which should put you in your home-directory on the NAS.

1. ls -ld to if all's well see a ~/.ssh directory with permissions drwx------ (700).
2. If so ls -l ~/.ssh to see an authorized_keys with permissions -rw------- (600)
3. If so cat ~/.ssh/authorized_keys to see the same key as you have on your computer in ~/.ssh/id_rsa.pub

If 1, 2 and 3 then simple ssh diskstation.local should not be prompting you for a password.

Per your edit: it's actually quite strange that /etc/ssh_host_rsa_key.pub on the NAS is equal to your ~/.ssh/id_rsa.pub on the computer. This would seem to indicate that the ssh-copy-id did something non-standard on the Synology. I.e, I expect that 1 above is already N/A? If so let's just do it manually.

4. ssh diskstation.local 'mkdir .ssh; chmod 700 .ssh'
5. scp ~/.ssh/id_rsa.pub diskstation.local:~/.ssh/authorized_keys
6. ssh diskstation.local 'chmod 600 .ssh/authorized_keys'

Then repeat 1, 2 and 3 to verify and log out. If definitely things now exist as per above then ssh diskstation.local should really not anymore prompt for a password. If still -- well, reboot doesn't hurt, but I doubt it'll help...

[AndyMH]

Working on it, did find this https://www.synoforum.com/threads/passw ... -ssh.5889/. Rusty's answer:

... edit the SSH service config : vim /etc/ssh/sshd_config
uncomment the lines PubkeyAuthentication yes and AuthorizedKeysFile .ssh/authorized_keys (make sure not to change anything else, otherwise you could lock yourself out of SSH)
restart the SSH service, either using synoservicectl --restart sshd or by disabling and re-enabling the SSH service in Control panel > Terminal &SNMP ...

Checked sshd_config and the two lines were commented out, have now un-commented them and restarted SSH, still wants a password, but I will follow your suggestions and post back.

[AndyMH]

ls -ld did not work, did not show any folders, had to use ls -la --group-directories-first to find that there is an .ssh folder.

Code: Select all

...
drwxrwxrwx+  2 andy  users       4096 Jun  2 19:02  .ssh
...

with content:

Code: Select all

andy@DiskStation:~/.ssh$ ls -l
total 4
-rwxrwxrwx+ 1 andy users 735 Jun  2 19:02 authorized_keys

Permissions are wrong and not checked content of authorized_keys - a job for tomorrow, too late now.

You said check permissions and in the link I found:

this is important: connect to the NAS by SSH and check the file's permissions:
chmod 0711 ~
chmod 0711 ~/.ssh
chmod 0600 ~/.ssh/authorized_keys

Hopefully that is the problem.

[AndyMH]

Last thing before bed:

Further down in the link I found

Any other hints on how to get keyfile ssh access working on DSM 7.0?

I've done this many times before on several older DSM models and a bunch of linux desktops, but can't get private keyfile access working on my new DS220+. Have followed the above instructions along with other similar threads on the web, without luck. Can log in using plain passwords just fine, but get a "server refused your key" when trying my private key file. Seems Synology have screwed around with sshd yet again on DSM 7.0

I'm running DSM 7.2 (and only upgraded from DSM 6 yesterday)

There are also comments in the link to suggest that the permissions do not matter.

More verbose output on connection:

Code: Select all

andy@M720 ~ $ ssh andy@diskstation.local -v
OpenSSH_8.9p1 Ubuntu-3ubuntu0.1, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to diskstation.local [192.168.0.5] port 22.
debug1: Connection established.
debug1: identity file /home/andy/.ssh/id_rsa type 0
debug1: identity file /home/andy/.ssh/id_rsa-cert type -1
debug1: identity file /home/andy/.ssh/id_ecdsa type -1
debug1: identity file /home/andy/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/andy/.ssh/id_ecdsa_sk type -1
debug1: identity file /home/andy/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /home/andy/.ssh/id_ed25519 type -1
debug1: identity file /home/andy/.ssh/id_ed25519-cert type -1
debug1: identity file /home/andy/.ssh/id_ed25519_sk type -1
debug1: identity file /home/andy/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /home/andy/.ssh/id_xmss type -1
debug1: identity file /home/andy/.ssh/id_xmss-cert type -1
debug1: identity file /home/andy/.ssh/id_dsa type -1
debug1: identity file /home/andy/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2
debug1: compat_banner: match: OpenSSH_8.2 pat OpenSSH* compat 0x04000000
debug1: Authenticating to diskstation.local:22 as 'andy'
debug1: load_hostkeys: fopen /home/andy/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:GRiEJopNcbTWgEdI3ZS3We1stSCars56By+X4n1t91I
debug1: load_hostkeys: fopen /home/andy/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'diskstation.local' is known and matches the ED25519 host key.
debug1: Found key in /home/andy/.ssh/known_hosts:3
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: agent returned 1 keys
debug1: Will attempt key: /home/andy/.ssh/id_rsa RSA SHA256:B1XfdkU6JrC4XB6V6Z69IWcX3yrvYGGYmt/AOzCgVRw agent
debug1: Will attempt key: /home/andy/.ssh/id_ecdsa 
debug1: Will attempt key: /home/andy/.ssh/id_ecdsa_sk 
debug1: Will attempt key: /home/andy/.ssh/id_ed25519 
debug1: Will attempt key: /home/andy/.ssh/id_ed25519_sk 
debug1: Will attempt key: /home/andy/.ssh/id_xmss 
debug1: Will attempt key: /home/andy/.ssh/id_dsa 
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering public key: /home/andy/.ssh/id_rsa RSA SHA256:B1XfdkU6JrC4XB6V6Z69IWcX3yrvYGGYmt/AOzCgVRw agent
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/andy/.ssh/id_ecdsa
debug1: Trying private key: /home/andy/.ssh/id_ecdsa_sk
debug1: Trying private key: /home/andy/.ssh/id_ed25519
debug1: Trying private key: /home/andy/.ssh/id_ed25519_sk
debug1: Trying private key: /home/andy/.ssh/id_xmss
debug1: Trying private key: /home/andy/.ssh/id_dsa
debug1: Next authentication method: password
andy@diskstation.local's password: 

[rene]

(yes, I meant to write ls -ld ~/.ssh)

Normally those permissions -- 700 for the directory, 600 for authorized_keys -- are in fact important. You can also just chmod them on the NAS if they exist there, and if latter matches your public key.

This should be straightforward.

[AndyMH]

Normally those permissions -- 700 for the directory, 600 for authorized_keys -- are in fact important.

chmoded .ssh and authorized_keys as above, no change, still asks for password. I'm going to delete .ssh on the NAS and do it all again manually following your previous guidance.

[rene]

I'm at this point skeptical you'd get it to work -- but supposedly also do a chmod 700 ~ while logged into the NAS.

Don't know why it's not working for you. It's fully standard SSH configuration and works between any two systems I've ever used, including to/from my own Netgear ReadyNAS boxes. Your above information as to /etc/ssh/ssh_host_rsa_key.pub implies that, weirdly, ssh-copy-id seems to have done something non-standard on Synology -- but anything I can find online says that at least the manual standard method should work fine with Synology as well.

If it really doesn't that's unfortunate but I've then no idea why not, foregoing weird configuration your-sides (which I don't expect since you said you only recently upgraded to DSM7, i.e., would supposedly remember if you configured something specific to have this now bomb out) and will then need to excuse myself from the thread.

[AndyMH]

Whatever it is, it is not the advice you've given me. If doing it again from scratch fails (which I expect) then it will be onto the synology forum and/or direct to synology tech support.

[AndyMH]

Sorted!!!!!

The final clue was here:
https://community.synology.com/enu/forum/1/post/136136

The weird thing is the file permissions. My home directory is on a BTRFS drive and Synology configures ACLs instead of using garden variety Linux permissions. The thing that finally makes it work is doing a "chmod 711 ." from my top-level directory on the NAS. This wipes out the ACLs and makes sshd happy.

I set up a new user on the NAS = backup (did not want to mess around with my main user account) and repeated previous guidance:

Code: Select all

ssh backup@diskstation.local 'mkdir .ssh; chmod 700 .ssh'
scp ~/.ssh/id_rsa.pub backup@diskstation.local:~/.ssh/authorized_keys
ssh backup@diskstation.local 'chmod 600 .ssh/authorized_keys'

Tested, it still wanted a password, so logged in:

Code: Select all

backup@DiskStation:~$ cd ..
backup@DiskStation:/var/services/homes$ ls -al
total 168
...
drwxrwxrwx+  4 backup    users   4096 Jun  4 19:19 backup
...

backup@DiskStation:/var/services/homes$ chmod 711 backup
backup@DiskStation:/var/services/homes$ ls -al
total 168
...
drwx--x--x   4 backup    users   4096 Jun  4 19:19 backup
...
backup@DiskStation:/var/services/homes$ exit

Then logged in again:

Code: Select all

andy@M720 ~ $ ssh backup@diskstation.local

Using terminal commands to modify system configs, execute external binary
files, add files, or install unauthorized third-party apps may lead to system
damages or unexpected behavior, or cause data loss. Make sure you are aware of
the consequences of each command and proceed at your own risk.

Warning: Data should only be stored in shared folders. Data stored elsewhere
may be deleted when the system is updated/restarted.

backup@DiskStation:~$ 

NO password prompt!

Rene, thanks for all your help on this. I would have struggled to get started without you, I then had to find out how synology was different. I suspect ACLs is down to win compatibility?

[rene]

The final clue was here:
https://community.synology.com/enu/forum/1/post/136136

The weird thing is the file permissions. My home directory is on a BTRFS drive and Synology configures ACLs instead of using garden variety Linux permissions. The thing that finally makes it work is doing a "chmod 711 ." from my top-level directory on the NAS. This wipes out the ACLs and makes sshd happy.

Good -- but also, sheesh...

I fairly much on purpose (originally) left out the chmod 700 from the originally linked tutorial -- "since that isn't necessary" -- and while it isn't normally indeed, well, goes to show; ACLs sticking their meddling noses in...

ACLs, Access Control Lists, provide for a New & Improved access control mechanism over old-school UNIX-permissions and do exists also on e.g. ext4;

Code: Select all

rene@hp8k:~$ touch foo
rene@hp8k:~$ getfacl foo
# file: foo
# owner: rene
# group: rene
user::rw-
group::r--
other::r--

rene@hp8k:~$ setfacl -m u:notrene:rw foo
rene@hp8k:~$ getfacl foo
# file: foo
# owner: rene
# group: rene
user::rw-
user:notrene:rw-
group::r--
mask::rw-
other::r--

Admittedly they're nicer than standard UNIX permissions in theory, but would in essence and/or normally seem to be implementable with use of UNIX-groups (which everyone in Linux seems to for some reason hate...) and in any case seem to when they live alongside regular UNIX-permissions in practice go unused, unless for those few specific instances where they horribly confuse matters such as seemingly in this case.

Yes, I do believe it's a more straight mapping onto e.g. NTFS also, but well, if you've read this forum for some time you know that at the very least here nobody in fact needs or wants -- or would be able to handle! -- any complication as to NTFS-permissions from Linux either.

I.e., all in all I never much gotten the (practical) point of ACLs and am sort of set on keeping it that way.

But good to know what the issue is/was. Hope things work as you wanted to now

[AndyMH]

The next thing to sort is ownership. rsyncing mint user andy to synology user backup ownership goes from uid 1000 to uid 1028 (synology users DO NOT start at 1000). Not bothered by that for the contents of home, but if, say, I rsync /etc, if I have to restore will it come back owned by root, I suspect not. More testing...