HelloItsME1212
Posts: 11
Joined: Tue Jun 12, 2018 4:11 pm

Rsync > Copy By folder name Keeping the directory structure

Sat Aug 04, 2018 8:55 pm

Okay so i have

/home/user/gnubiedirectory


I want to rsync every change in real time(or every 1 min) thath happens in thath directory to my raspberry.

But i want to copy just folders thath are named "logs" and all the files inside those folders obv.


Also i need to maintain the folder structure. For example

/home/user/gnubiedirectory/ubuntu/random/logs/(9 text files inside)


in my raspberry should copy

gnubiedirectory/ubuntu/random/logs/


i mean, each logs folder in each random named folder inside each ubuntu named folder. just copy logs+what is inside logs but copying all folders thath are above.

DirkS
Posts: 9038
Joined: Tue Jun 19, 2012 9:46 pm
Location: Essex, UK

Re: Rsync > Copy By folder name Keeping the directory structure

Sat Aug 04, 2018 10:41 pm

Have a look at rsync's filter rules (see e.g. https://linux.die.net/man/1/rsync)

achrn
Posts: 311
Joined: Wed Feb 13, 2013 1:22 pm

Re: Rsync > Copy By folder name Keeping the directory structure

Sun Aug 05, 2018 8:13 am

I think what you want is

Code: Select all

rsync -r -m --include='**/' --include='**/logs/*' --exclude='**' /home/user/gnubiedirectory wherever-you-want-it
The filters say:
  1. include every directory (but just the directory, no content)
  2. include every file inside a directory called logs
  3. exclude everything
The first filter to match decides what happens, so rule 3 is implicitly 'exclude everything else'. Thus, this will transfer the whole directory structure (but it will all be empty, except the 'logs' directories which have their content), but the '-m' option then ignores any empty directories (ie, all those that are not called 'logs'). Note that if you're sending this across the network, I think it does actually send all the empty directories, and the receiving end discards the empties, so it potentially has a network traffic implication.

If you want the whole tree below 'logs' directories you probably want

Code: Select all

rsync -r -m --include='**/' --include='**/logs/***' --exclude='**' /home/user/gnubiedirectory wherever-you-want-it
That is, if you had
blaah/blaah/logs/fred.txt
blaah/blaah/logs/wilma.txt
blaah/blaah/logs/rubbles/barney.txt
blaah/blaah/logs/rubbles/betty.txt

The first option would give you fred.txt and wilma.txt, but the second would get you fred.txt, wilma.txt, rubbles/barney.txt and rubbles/betty.txt

HelloItsME1212
Posts: 11
Joined: Tue Jun 12, 2018 4:11 pm

Re: Rsync > Copy By folder name Keeping the directory structure

Sun Aug 05, 2018 8:36 am

I Found another issue, if i need to sincronize files from my Server to raspberry, and raspberry right now isnt open for the world(is just accesible from my network) i need to open ports so the server can reach the raspberry right?

@achrn thanks

achrn
Posts: 311
Joined: Wed Feb 13, 2013 1:22 pm

Re: Rsync > Copy By folder name Keeping the directory structure

Sun Aug 05, 2018 1:07 pm

HelloItsME1212 wrote:
Sun Aug 05, 2018 8:36 am
I Found another issue, if i need to sincronize files from my Server to raspberry, and raspberry right now isnt open for the world(is just accesible from my network) i need to open ports so the server can reach the raspberry right?
You either need the server to be able to initialise the connection (and thus require open ports on the raspberry) or you need the raspberry to be able to open a connection to your server (and thus require open ports on the server).

My backup regime uses a Raspberry that uses rsync to pull selected stuff from all the unix-like machines on my network. The Raspberry initialises the connection , i.e. it's the Raspberry that runs an rsync, and the other machines permit the raspberry to connect. All my other machines allow the relevant connection from the Raspberry, and have authorized_keys files so that the raspberry is authenticated without passwords etc. So technically the Raspberry doesn't need open ports - but one or other machine does require it.

Your reference to 'world' suggests this is across the internet. You need to be a whole lot more careful in that case - my system only runs on my lan, on the inside of my firewall.

HelloItsME1212
Posts: 11
Joined: Tue Jun 12, 2018 4:11 pm

Re: Rsync > Copy By folder name Keeping the directory structure

Mon Aug 06, 2018 5:15 pm

achrn wrote:
Sun Aug 05, 2018 8:13 am
I think what you want is

Code: Select all

rsync -r -m --include='**/' --include='**/logs/*' --exclude='**' /home/user/gnubiedirectory wherever-you-want-it
The filters say:
  1. include every directory (but just the directory, no content)
  2. include every file inside a directory called logs
  3. exclude everything


Using the first command

Code: Select all

[email protected]:~$ rsync -r -v -P -m --include='**/logs/*' --exclude='**' /home/user/Tribal/ [email protected] REAL IP
building file list ...
1 file to consider

sent 36 bytes  received 12 bytes  96.00 bytes/sec
total size is 0  speedup is 0.00
[email protected]:~$
and it should be more than 45000 archives I will add some screenshots so you can see better what needs to be copy and where is

Image
Image

what here needs to be copy, it also needs to be copy in the structure of folders is the same, and what to copy also it is
~/Tribal/es51
~/Tribal/br90
~/Tribal/de157
~/Tribal/pt62

achrn
Posts: 311
Joined: Wed Feb 13, 2013 1:22 pm

Re: Rsync > Copy By folder name Keeping the directory structure

Sun Aug 12, 2018 8:37 am

HelloItsME1212 wrote:
Mon Aug 06, 2018 5:15 pm
achrn wrote:
Sun Aug 05, 2018 8:13 am
I think what you want is

Code: Select all

rsync -r -m --include='**/' --include='**/logs/*' --exclude='**' /home/user/gnubiedirectory wherever-you-want-it
The filters say:
  1. include every directory (but just the directory, no content)
  2. include every file inside a directory called logs
  3. exclude everything


Using the first command

Code: Select all

[email protected]:~$ rsync -r -v -P -m --include='**/logs/*' --exclude='**' /home/user/Tribal/ [email protected] REAL IP
building file list ...
1 file to consider

sent 36 bytes  received 12 bytes  96.00 bytes/sec
total size is 0  speedup is 0.00
[email protected]:~$
and it should be more than 45000 archives I will add some screenshots so you can see better what needs to be copy and where is
That's not the command I suggested.

Your version won't descend the directory tree, because all the directories at the highest level (except any called logs) are excluded, so never get looked in. You need all three of the filters I suggested.

Return to “Networking and servers”

Who is online

Users browsing this forum: No registered users and 9 guests