User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Does cloning an SD clone the Pi's MAC address?

Mon Apr 15, 2019 4:22 pm

I don't care about the MAC address, except my DHCP server (Android phone) puts them both on the same IP address, which doesn't work very well.

I cloned the SD in my upstairs Pi and put it in my downstairs Pi. All of a sudden the internet seems terribly slow. But only on the Pi, my phone and laptop are fine. I was downstairs and happened to do an ifconfig, and I have 2 Pis on 192.168.43.222. So I turned the downstairs one back off. Turned it back on later, same IP. I haven't compared the MAC addresses, I just remember the 222. Seems like I remember in some cases MAC addresses can be set by software. That would be with the Pi's internal wifi, I have a USB one with a gain antenna I should be using down there, but I have to remember how to switch it over. Ignore wlan0 and use wlan1, somehow. I did it last summer.

User avatar
DougieLawson
Posts: 36308
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Does cloning an SD clone the Pi's MAC address?

Mon Apr 15, 2019 4:43 pm

MAC address is set from the bottom 24-bits of the CPU serial.

WiFi MAC address is set from the bottom 24-bits of the CPU serial XOR'd with 0x555555
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Mon Apr 15, 2019 5:49 pm

Interesting. I wrote the MAC address from the upstairs one on paper, brought it downstairs and compared, last part is different. So the Android DHCP server is only looking at the first (manufacturer) part of the MAC. That seems like a big security hole. And for anybody with a bunch of identical hardware a nuisance.

I use a very manual interfaces file:

auto wlan1
iface wlan1 inet dhcp
wireless-essid Moto_lte

So I just changed wlan 0 to wlan1 and I'm on the Realtek. I didn't have the duplicate IP problem last summer.

epoch1970
Posts: 3783
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Does cloning an SD clone the Pi's MAC address?

Mon Apr 15, 2019 7:27 pm

It seems unlikely that the DHCP server only uses 3 bytes to match an IP.
Systemd has made the role of /etc/machine-id (a D-Bus thing) more prominent, perhaps your DHCP client uses that ID when it talks to the server.

Of course, there are special motions to go through in case you want to re-generate /etc/machine-id.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

User avatar
lpsw
Posts: 157
Joined: Thu Jun 25, 2015 4:45 pm
Location: USA

Re: Does cloning an SD clone the Pi's MAC address?

Mon Apr 15, 2019 7:38 pm

epoch1970,
This is off topic, just want to express our sorrow and heavy hearts for Paris and all of France for today's historic disaster.
Vive La France.

EDIT: fixed my bad spelling.
Self-education is, I firmly believe, the only kind of education there is - Isaac Asimov

epoch1970
Posts: 3783
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Does cloning an SD clone the Pi's MAC address?

Mon Apr 15, 2019 7:58 pm

lpsw wrote:
Mon Apr 15, 2019 7:38 pm
This is off topic, just want to express our sorrow and heavy hearts for Paris and all of France for today's historic disaster.
Vive La France.
Thanks. I'm actually not too far from Notre Dame. Not going out to watch the disaster tonight but tomorrow the cityscape won't be too pretty I'm afraid.
Let's hope nothing too irreplaceable has gone up in flames; For the rest, it's been built, we will rebuild.
EDIT. Hum. It's reported one of the belfries is at risk of catching on fire. The roof/attic is one thing, but a tower...
(end of of the OT)
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

Andyroo

Re: Does cloning an SD clone the Pi's MAC address?

Mon Apr 15, 2019 9:26 pm

Hope everyone is safe.

As for the OP duplicate MAC address - not sure if this is the same issue or not viewtopic.php?f=66&t=238170

User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Tue Apr 16, 2019 6:37 pm

epoch1970 wrote:
Mon Apr 15, 2019 7:27 pm
It seems unlikely that the DHCP server only uses 3 bytes to match an IP.
Systemd has made the role of /etc/machine-id (a D-Bus thing) more prominent, perhaps your DHCP client uses that ID when it talks to the server.

Of course, there are special motions to go through in case you want to re-generate /etc/machine-id.
That's the other possibility. I was cloning SDs a year or so ago and this never happened. I used piclone. Maybe it should leave out /etc/machine-id or generate a new one, like it can generate new UUIDs.

But would that get passed to the DHCP server? Hmm, there's an /etc/dhcpd.duid which contains the MAC address, that would have gotten cloned too. And /etc/dhcpcd.secret . But a DHCP server can usually read a MAC address directly. Oh well, if nobody else is having a problem with it...

No, the MAC address didn't get cloned, the DHCP server just put the cloned machine on the same IP as the original, for some reason. This machine's been on that IP for months, the 222 is easy to remember.

Notre Dame: wasn't there a bunch of unique art at risk of getting destroyed or water damaged?

epoch1970
Posts: 3783
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Does cloning an SD clone the Pi's MAC address?

Tue Apr 16, 2019 11:25 pm

The subject of DHCP client identification is protracted because it mixes DHCP-ipv4, traditionally relying on MAC address and that curious beast that is DHCPv6 which introduced the notion of machine-specific "DHCP UID". There are different flavors of DUIDs and every DHCP client handles either identification options in their own way.
Even in case DHCPv6 doesn't finally meet the success it doesn't deserve, DUID has some attraction because with hardware virtualisation MACs are not as solid as they used to be. So DHCP clients tend to use DUIDs, even over ipv4.

In short if you're cloning, don't clone /etc/machine-id, bad things could happen. Don't clone DUID files or past DHCP leases files either. The location of these files and the generation procedure depends on the DHCP client.


Re: Notre Dame, yes there was a few trinkets there. Nothing of great importance has perished it seems, some paintings and stained glass pieces suffered from fumes but neither from fire nor water. The spire that went down was less than 200 years old, the original one had been destroyed during the Revolution.
All firefighters accounted for. An electrical fire on the platform installed for the renovation of the spire is the likely cause.
Now we just need to find a medieval forest of oak trees in order to rebuild the roof. The forest they came from is long gone...
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Wed Apr 17, 2019 2:27 am

Well, I didn't tell piclone what to copy or not, it just did everything. I wonder if the people who maintain piclone (in the Raspbian debs) should adjust it. It lives at https://github.com/raspberrypi-ui/piclo ... master.zip maybe I should file an issue there.

I don't really like ipv6, try not to use it. I'm behind my ISP's firewall and using NAT anyway, it serves no purpose here.

PhatFil
Posts: 1411
Joined: Thu Apr 13, 2017 3:55 pm
Location: Oxford UK

Re: Does cloning an SD clone the Pi's MAC address?

Wed Apr 17, 2019 2:43 am

Im sure i have come across a Pi 'How to' that details a method of specifying the mac addy to use, of course i have failed in my google-fu to find it to link .. I can only assume that if my recall is correct and such a 'fix' is Pi applicable, Then in such a case where a user specified Mac addy is applied I would suggest that Yes The mac addy would also be duplicated with a system disk clone since it Must be applied through a file.

User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Wed Apr 17, 2019 3:23 am

PhatFil wrote:
Wed Apr 17, 2019 2:43 am
Im sure i have come across a Pi 'How to' that details a method of specifying the mac addy to use, of course i have failed in my google-fu to find it to link .. I can only assume that if my recall is correct and such a 'fix' is Pi applicable, Then in such a case where a user specified Mac addy is applied I would suggest that Yes The mac addy would also be duplicated with a system disk clone since it Must be applied through a file.
That's what I don't like about setting it from a config file. Used to be, it was in the hardware of the interface card and not changeable.

I've side-stepped my problem by using a USB wifi adapter, which I was going to use anyway because this Pi is at considerable distance from the AP.

User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Thu Apr 18, 2019 12:57 am


User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Thu Apr 18, 2019 1:45 pm

Well, that's a drag
The purpose of the SD card cloner, as the name suggests, is to completely copy the original card; to create a clone for backup purposes. This may well lead to problems if you try and run 2 Pis with identical cards on the same network, but equally it may well lead to problems if we try to modify someone's image which they are relying on being a drop-in backup of their existing card.

The program is designed to create an exact backup - if you want a card for a second Pi, you will need to make whatever adjustments are required yourself, I'm afraid.
I was hoping they might dig into it and provide an option to generate new DHCP UUIDs like they do for partition UUIDs. Maybe somebody (not me) will create a fork that does that. I didn't know it was mostly for backups, I've been using it in place of downloading a fresh image when I get a new machine. For years.

A year or so ago I bought an Android phone, of the same model as an earlier one I still have. Because Google keeps a database of what apps you have on your phone all the same apps got installed on the second one. Convenient but spooky.

And in the Pi case I usually keep around stuff I've built from source, either because it isn't in the Raspbian debs or because the deb version is too old to be useful. Alright, somebody's gotta come up with something to make these unique enough again. The same thing would happened if you copied using dd.

Somebody made an assumption, the designers of the DHCP stuff assumed nobody would clone a card. I think it's a shortcoming in the way DHCP is done here. I wonder if you can remove DHCP and reinstall it. DHCP has existed and worked for years on other things just fine. This breaks the Linux usage of it.

It's a little like here in the US a Social Security number was just something most people had, there wasn't anything confidential about it,. Then some lazy banks started using it as proof of who you were, so it becomes a closely guarded secret. Which is the banks fault, it's still a useful unique number (think primary key in an SQL sense).

User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Thu Apr 18, 2019 3:20 pm

So I answered:

Well, yes, if I cloned an SD with dd I might expect that to happen. But since Piclone has the option of generating new DUIDs (or UUIDs, or whatever else you want to call them, I think they're silly), I was hoping you might add that option for this.

It becomes a political problem of whose fault it is, and since DHCP breaks if you clone an SD (by whatever method) I'd say it's the DHCP people that need to fix it. It's counterproductive for every end user to try to fix it individually because we don't have the knowledge or connections and we'd be reinventing the wheel over and over. Or we'd need a how-to from an authoritative source at least.

It's of significant benefit to be able to just clone a card and update/upgrade it rather than starting from scratch especially if you use/build/write software that isn't in the debs so it's not easily reinstalled. I think I downloaded one copy of Stretch and cloned it, until I ran into this time bomb. What happens if I use mkelfimage I wonder. These foolish UUIDs need to be regenerated and there should be a better way of causing that to happen. Hopefully not manually.

-------------
OK, mkelfimage is only for i386, I thought I'd seen something like that though.

User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Thu Apr 18, 2019 5:34 pm

They locked up the Github conversation. I'm thinking Clonezilla might be better to use https://clonezilla.org/ but I don't know yet about what non-disk stuff it handles like UUIDs and ssh keys. It's in Debian's debs, haven't checked Raspbian's. It attempts to do what Ghost can do, it does deployment onto other machines.

User avatar
DougieLawson
Posts: 36308
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Does cloning an SD clone the Pi's MAC address?

Thu Apr 18, 2019 5:56 pm

You've got the source code for piclone, hack it to do the stuff you want like generating new SSH keys (which simply needs the files to be removed and they'll get created automagically at boot time), hostname (set to RaspberryPi unless you specify it), etc.

If you get something that passes muster with Simon Long he might accept a pull request (rather than a bury-head-in-sand request). Piclone should have some boxes on the menu for "Is this a backup for this Raspberry? [Y/N]" "Is this a copy for a different machine? [Y/N]" (obviously mutually exclusive).
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Thu Apr 18, 2019 6:42 pm

Thanks, it's a possibility. Easier to just use a different wifi card.

Hostname, yes, I always change those. Mostly without other tinkering that makes the hostname visible in DNS on your LAN or over wifi. On my phone I can get my Dump1090 page from my zero3 for example.

ssh key, that's ssh-keygen

It's the dbus and other stuff I don't want to get into. And Pis are different (with more of it) than this Debian laptop, but I don't clone this.

Code: Select all

d530# cd /etc
d530# ls -lR | grep -i dhcp
drwxr-xr-x  4 root root     4096 Nov 18 19:25 dhcp
./dhcp:
./dhcp/dhclient-enter-hooks.d:
./dhcp/dhclient-exit-hooks.d:

uname -a
Linux d530 4.9.0-7-amd64 #1 SMP Debian 4.9.110-3+deb9u2 (2018-08-13) x86_64 GNU/Linux
Too much badly thought-out stuff on Pis:

Code: Select all

drwxr-xr-x  4 root root    4096 Nov 13 08:04 dhcp
-rw-rw-r--  1 root netdev  1701 Sep 10  2018 dhcpcd.conf
-rw-r--r--  1 root root      42 Nov 13 09:02 dhcpcd.duid
-r--------  1 root root     192 Nov 13 09:02 dhcpcd.secret
lrwxrwxrwx 1 root root  13 Nov 13 08:22 dhcpcd -> /sbin/dhcpcd5
lrwxrwxrwx 1 root root  32 Nov 13 08:22 dhcpcd.8.gz -> /usr/share/man/man8/dhcpcd5.8.gz
./dhcp:
./dhcp/dhclient-enter-hooks.d:
./dhcp/dhclient-exit-hooks.d:
-rwxr-xr-x 1 root root 1901 Sep 14  2015 dhcpcd
lrwxrwxrwx 1 root root 16 Nov 13 08:22 K01dhcpcd -> ../init.d/dhcpcd
lrwxrwxrwx 1 root root 16 Nov 13 08:22 K01dhcpcd -> ../init.d/dhcpcd
lrwxrwxrwx 1 root root 16 Nov 13 08:22 S01dhcpcd -> ../init.d/dhcpcd
lrwxrwxrwx 1 root root 16 Nov 13 08:22 S01dhcpcd -> ../init.d/dhcpcd
lrwxrwxrwx 1 root root 16 Nov 13 08:22 S01dhcpcd -> ../init.d/dhcpcd
lrwxrwxrwx 1 root root 16 Nov 13 08:22 S01dhcpcd -> ../init.d/dhcpcd
lrwxrwxrwx 1 root root 16 Nov 13 08:22 K01dhcpcd -> ../init.d/dhcpcd
-rw-r--r-- 1 root root    0 Apr 20 09:03 dhcp.conf
drwxr-xr-x 2 root root 4096 Nov 13 08:22 dhcpcd.service.d
lrwxrwxrwx 1 root root   34 Nov 13 08:22 dhcpcd5.service -> /lib/systemd/system/dhcpcd.service
./systemd/system/dhcpcd.service.d:
lrwxrwxrwx 1 root root 34 Nov 13 08:22 dhcpcd.service -> /lib/systemd/system/dhcpcd.service
I like Pis, but this is almost a vulnerability.

spl23
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 378
Joined: Fri Dec 26, 2014 11:02 am

Re: Does cloning an SD clone the Pi's MAC address?

Sun Apr 21, 2019 11:58 am

DougieLawson wrote:
Thu Apr 18, 2019 5:56 pm
Piclone should have some boxes on the menu for "Is this a backup for this Raspberry? [Y/N]" "Is this a copy for a different machine? [Y/N]" (obviously mutually exclusive).
A strange usage of the word "should" there, given that it was me who wrote Piclone. So I think it's probably up to me to decide what it *should* do, not you, don't you?

A complicated tool to handle multiple situations will fail to handle every single situation which is thrown at it, because of the sheer variety of changes people make to Pi images. One trivial example is a card with a static IP address - you clearly shouldn't reuse the same one. So what do you change it to? Or do you propose that said tool searches the local network, finds an apparently unused address, and assigns it for you? Because there's no way that could ever go wrong...

If you do want a more complex tool, feel free to write one yourself. But that is not what Piclone is for, and it is not something Piclone will be changed to do. Piclone clones a card; the clue is in the name.

User avatar
ab1jx
Posts: 868
Joined: Thu Sep 26, 2013 1:54 pm
Location: Heath, MA USA
Contact: Website

Re: Does cloning an SD clone the Pi's MAC address?

Sun Apr 21, 2019 1:37 pm

spl23 wrote:
Sun Apr 21, 2019 11:58 am
DougieLawson wrote:
Thu Apr 18, 2019 5:56 pm
Piclone should have some boxes on the menu for "Is this a backup for this Raspberry? [Y/N]" "Is this a copy for a different machine? [Y/N]" (obviously mutually exclusive).
Maybe I wrote "should" too.

I meant should in the sense of "would be useful if". Since Piclone is the Rasbian debs and on the https://github.com/raspberrypi-ui site it seems fairly official. Is this not the recommended tool for deployment? Especially since the Pi has its origins in education I can imagine classrooms full of them. I was involved in doing that with Windows machines for over 10 years in education. There might be duplicators (hardware) for SD cards, you could have sets of SD cards prepared for different classes, that sort of thing. It's like having swappable hard drives. Because they're much cheaper than hard drives it's more feasible for each student to have their own. We could reload a machine over the network, or in a different situation from an image we'd made and kept on DVD. And what was on the machines frequently got trashed so they needed reloading.

What seems to be problematic here are things like UUIDs, DUIDs, keys, dbus stuff that need to be unique. Because there seems to be more of that on a Pi than in Debian the most likely place to find a cloning tool seems from the Raspberry Pi foundation, where inside knowledge exists.

I think I have used it on non-Pi machines, so it should be a checkbox, not something that's always on. But regenerating things that need to be unique is a necessary part of the cloning process.

User avatar
rin67630
Posts: 568
Joined: Fri Mar 04, 2016 10:15 am

Re: Does cloning an SD clone the Pi's MAC address?

Thu Aug 15, 2019 7:09 am

ab1jx wrote:
Mon Apr 15, 2019 4:22 pm
I don't care about the MAC address, except my DHCP server (Android phone) puts them both on the same IP address, which doesn't work very well.
I am cloning hundreds of SD cards and can say that piclone has nothing to do with WLAN MAC addresses.

These are tied to the WLAN chip used: on RPi3 with WLAN the MAC address differs for every board.
If you use an external USB to WLAN dongle, that one will determine the MAC address. I have never seen USB to WLAN dongles with the same MAC (but one never know what crap the chinese may produce)

If you get the same IP from your DHCP server with different hardware, your DHCP server is crap.
Android never was a good WLAN hub, I'd use something like the very affordable T-Link MR3020 which can provide WISP services (use a WLAN as Internet Service provider).

Return to “Troubleshooting”