java_c
Posts: 3
Joined: Tue Aug 03, 2021 6:54 pm

Running out of available space

Tue Aug 03, 2021 7:22 pm

Hi, two months ago I upgraded my raspberry from a 16GB to a 32 GB card.
After copying the card I expanded the filesystem via raspi-config "expand Filesystem".

This work's fine and I hat the available space of about 15GB.


Now two months later I ran again in the situation that I run out of space. I already searched for a solution but couldn't found a solution.

Please find here some details:
Available DiskSpace

Code: Select all

sudo df

Dateisystem          1K-Blöcke   Benutzt  Verfügbar Verw% Eingehängt auf
/dev/root             30613684  30544556          0  100% /
devtmpfs                469544         0     469544    0% /dev
tmpfs                   474152         0     474152    0% /dev/shm
tmpfs                   474152     12292     461860    3% /run
tmpfs                     5120         8       5112    1% /run/lock
tmpfs                   474152         0     474152    0% /sys/fs/cgroup
none                      5120       180       4940    4% /var/log
/dev/mmcblk0p1           42131     22856      19275   55% /boot
//192.168.2.30/home 1917962728 813937064 1103906880   43% /NAS/pi
tmpfs                    94828         0      94828    0% /run/user/1000
Available Inodes

Code: Select all


sudo df -i

Dateisystem            Inodes IBenutzt     IFrei IUse% Eingehängt auf
/dev/root             1877344   131868   1745476    8% /
devtmpfs               117386      390    116996    1% /dev
tmpfs                  118538        1    118537    1% /dev/shm
tmpfs                  118538      446    118092    1% /run
tmpfs                  118538        4    118534    1% /run/lock
tmpfs                  118538       12    118526    1% /sys/fs/cgroup
none                   118538       12    118526    1% /var/log
/dev/mmcblk0p1              0        0         0     - /boot
//192.168.2.30/home 121798656   595318 121203338    1% /NAS/pi
tmpfs                  118538       11    118527    1% /run/user/1000

Code: Select all

sudo fdisk -l
...
Disk /dev/mmcblk0: 29,7 GiB, 31914983424 bytes, 62333952 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc92dee4d

Device         Boot Start      End  Sectors  Size Id Type
/dev/mmcblk0p1       8192    93802    85611 41,8M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      98304 62333951 62235648 29,7G 83 Linux

Code: Select all

sudo ncdu
  118,9 GiB [##########] /NAS
    1,7 GiB [          ] /usr
  368,2 MiB [          ] /opt
  201,0 MiB [          ] /var
  200,0 MiB [          ] /home
  199,6 MiB [          ] /root
  191,2 MiB [          ] /lib
   22,3 MiB [          ] /boot
   12,0 MiB [          ] /run
    7,4 MiB [          ] /sbin
    6,7 MiB [          ] /bin
    4,0 MiB [          ] /etc
   32,0 KiB [          ] /tmp
e  16,0 KiB [          ] /lost+found
    8,0 KiB [          ] /data
e   4,0 KiB [          ] /srv
e   4,0 KiB [          ] /mnt
e   4,0 KiB [          ] /media
e   4,0 KiB [          ] /Synology
    0,0   B [          ] /sys
.   0,0   B [          ] /proc
    0,0   B [          ] /dev

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29346
Joined: Sat Jul 30, 2011 7:41 pm

Re: Running out of available space

Wed Aug 04, 2021 8:44 am

You might have some massive log files taking up space?

Check /var/log

Also, there is a disk usage utility you can install, that will tell you what is using up space where, which might help track things down.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Applications Team.

User avatar
RaTTuS
Posts: 10790
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: Running out of available space

Wed Aug 04, 2021 8:53 am

du -h / | sort -nr | less

but maybe you have filled up
/NAS/pi
by stuffing stuff there before it is loaded from the network
unount that drive and look in there
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

cleverca22
Posts: 4385
Joined: Sat Aug 18, 2012 2:33 pm

Re: Running out of available space

Wed Aug 04, 2021 10:05 am

first tip, always use "ncdu -x", so it wont count space from a remote machine (your nas)
2nd tip along the lines of what RaTTuS said, temporarily `umount /NAS/pi` so ncdu can look behind that curtain, your likely writing stuff when it isnt mounted

jbudd
Posts: 1893
Joined: Mon Dec 16, 2013 10:23 am

Re: Running out of available space

Wed Aug 04, 2021 10:09 am

The only time my Pies have run out of space it was huge logfiles in /var/log.

The du command will help track down where your disk is filling up
sudo du -sm /* 2>/dev/null | sort -V shows the disk usage in MB of each directory in /

Code: Select all

sudo du -sm /* 2>/dev/null | sort -V
0       /bin
0       /lib
0       /proc
0       /sbin
0       /sys
1       /lost+found
1       /media
1       /mnt
1       /srv
1       /tmp
2       /dev
9       /run
22      /etc
47      /opt
49      /boot
49      /boot.bak
58      /root
750     /var
2561    /home
6404    /usr
Followed by (eg) sudo du -sm /var/* 2>/dev/null | sort -V

java_c
Posts: 3
Joined: Tue Aug 03, 2021 6:54 pm

Re: Running out of available space

Mon Aug 09, 2021 8:19 am

First of all thanks for your feedback:
- I disabled cifs mounting of network folder in /etc/fstab

Code: Select all

//192.168.2.30/home /NAS/pi cifs username=*****,password=*****,workgroup=workgroup,vers=1.0,users,auto,user_xattr 0 0
I found that two backup files of my monthly backup were stored directly on the ssd card and not on the network folder /NAS/pi.
Deleting these files solved my problem.

Is there a way to protect such behaviour?

Here is the script I run once a month.

Code: Select all

BACKUP_PATH="/NAS/pi"
BACKUP_NAME="RaspberryPiBackup"
dd if=/dev/mmcblk0 of=${BACKUP_PATH}/${BACKUP_NAME}-$(date +%Y%m%d-%H%M%S).img bs=1MB

User avatar
rpdom
Posts: 19044
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Running out of available space

Mon Aug 09, 2021 8:26 am

java_c wrote:
Mon Aug 09, 2021 8:19 am
Is there a way to protect such behaviour?
A simple check could be:

Code: Select all

if ! mountpoint -q ${BACKUP_PATH}
then
  echo "Backup drive is not mounted!"
  exit 1
fi
Unreadable squiggle

User avatar
thagrol
Posts: 5558
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Running out of available space

Mon Aug 09, 2021 11:12 am

rpdom wrote:
Mon Aug 09, 2021 8:26 am
java_c wrote:
Mon Aug 09, 2021 8:19 am
Is there a way to protect such behaviour?
A simple check could be:

Code: Select all

if ! mountpoint -q ${BACKUP_PATH}
then
  echo "Backup drive is not mounted!"
  exit 1
fi
And/or unmount the NAS then

Code: Select all

chmod a-w /NAS/pi
That'll prevent anyone/anything writing when the NAS is not mounted but the errors will be more obscure.

Though frankly, unless your backup process is running as root I'm surprised you can write to the NAS with that fstab entry. You don't set either local user/group or local permissions so will be getting the default (root:root rwxr-xr-x)

And why have you included the users mount option? Do you really want any random user on your system to be able to mount and unmout the NAS?
I'm a volunteer. Take me for granted or abuse my support and I will walk away

All advice given is based on my experience. it worked for me, it may not work for you.
Need help? https://github.com/thagrol/Guides

cleverca22
Posts: 4385
Joined: Sat Aug 18, 2012 2:33 pm

Re: Running out of available space

Mon Aug 09, 2021 1:14 pm

java_c wrote:
Mon Aug 09, 2021 8:19 am
Is there a way to protect such behaviour?
my choice, is to `chmod 0 /NAS/pi` while its not mounted, so nobody has permission to do anything

then do what others have said, and test that its actually mounted, the chmod is just a safety net, so it will fail if you dont test right

java_c
Posts: 3
Joined: Tue Aug 03, 2021 6:54 pm

Re: Running out of available space

Tue Aug 10, 2021 6:14 am

Thanks to all of you, I could handle the issue with the provided script. I included this in my monthly backup procedure.
Also many thanks for the hint regarding users in cifs.

Return to “Troubleshooting”