Monday, June 15, 2015

sfdisk --json

As promised by the previous blog I have merged support for JSON dump to sfdisk/libfdisk.
# sfdisk --json /dev/sda
{
   "partitiontable": {
      "label": "gpt",
      "id": "E471810B-5954-48E6-B4F0-5D369ADCC514",
      "device": "/dev/sda",
      "unit": "sectors",
      "firstlba": 34,
      "lastlba": 468862094,
      "partitions": [
         {"node": "/dev/sda1", "start": 2048, "size": 409600, "type": "C12A7328-F81F-11D2-BA4B-00A0C93EC93B", "uuid": "2062335E-51B6-49E8-BC3C-4C6D449E6805", "name": "EFI System Partition"},
         {"node": "/dev/sda2", "start": 411648, "size": 409600, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "DF378D5F-B7BA-4887-BAEB-A81E48DE903D"},
         {"node": "/dev/sda3", "start": 821248, "size": 273266688, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "6073277F-87BC-43FF-BCFD-724C4484A63A"},
         {"node": "/dev/sda4", "start": 274087936, "size": 104857600, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "EC5A52E0-FE99-4B38-91E7-B5FF1120E0E5"},
         {"node": "/dev/sda5", "start": 378945536, "size": 73531392, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "11A62EAE-5ECA-4EF5-8BF5-DD18A4115F4A"},
         {"node": "/dev/sda6", "start": 452476928, "size": 16384000, "type": "0657FD6D-A4AB-43C4-84E5-0933C84B4F4F", "uuid": "A13DACFE-6FFB-4B2C-BD89-B91B768A4F77"}
      ]
   }
}

Friday, June 5, 2015

JSON output for basic tools

I have merged into libsmartcols support for JSON format. The feature is currently enabled for lsblk(8), findmnt(8), lslocks(8) and losetup(8). The library already uses structured data, so add a different output format is pretty trivial. For example findmnt(8) and lsblk(8) are able to gather much information and standardized parsable format makes it easy to use it monitoring tools, automatic bug reporting tools, etc.
$ lsblk --json -o name,type,mountpoint                                    
{                                                                         
   "blockdevices": [                                                      
      {"name": "sda", "type": "disk", "mountpoint": null,                 
         "children": [                                                    
            {"name": "sda1", "type": "part", "mountpoint": "/boot/efi"},  
            {"name": "sda2", "type": "part", "mountpoint": "/boot"},      
            {"name": "sda3", "type": "part", "mountpoint": "/home"},      
            {"name": "sda4", "type": "part", "mountpoint": "/"},          
            {"name": "sda5", "type": "part", "mountpoint": "/home/wine"}, 
            {"name": "sda6", "type": "part", "mountpoint": "[SWAP]"}      
         ]                                                                
      },                                                                  
      {"name": "sdb", "type": "disk", "mountpoint": null,                 
         "children": [                                                    
            {"name": "sdb1", "type": "part", "mountpoint": "/home/archive"}
         ]                                                                
      }                                                                   
   ]                                                                      
}                                                                         
                                                                    
$ findmnt --json --submounts /home                                        
{                                                                         
   "filesystems": [                                                       
      {"target": "/home", "source": "/dev/sda3", "fstype": "ext4", "options": "rw,relatime,data=ordered",
         "children": [                                                    
            {"target": "/home/wine", "source": "/dev/sda5", "fstype": "ext4", "options": "rw,relatime,data=ordered"},
            {"target": "/home/archive", "source": "/dev/sdb1", "fstype": "ext4", "options": "rw,relatime,data=ordered"}
         ]                                                                
      }                                                                   
   ]                                                                      
}
For the next Monday my TODO contains "JSON sfdisk/libfdisk dump output"...

Wednesday, May 6, 2015

resize by sfdisk

The last week released util-linux v2.26.2 contains improved (fixed:-) sfdisk. The most visible change is that the new version supports partition resize and start offset move.

For example 1GiB disk with GPT:

        echo ", +10M" | ./sfdisk -N 1 /dev/sdc
        ...
       
        Old situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 206847  204800  100M Linux filesystem
       
        New situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 227327  225280  110M Linux filesystem
the first partition (-N 1) has been enlarged from 100MiB to 110MiB.

You can also move begin of the partition:

        echo "+2M" | ./sfdisk -N 1 /dev/sdc
        ...
        Old situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 227327  225280  110M Linux filesystem
       
        New situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   6144 231423  225280  110M Linux filesystem
and it's possible to enalarge as much as possible:
       
        echo ", +" | ./sfdisk -N 1 /dev/sdc
        ...
        Old situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   6144 231423  225280  110M Linux filesystem
       
        New situation:
 
        Device     Start     End Sectors  Size Type
        /dev/sdc1   6144 2047966 2041823  997M Linux filesystem
the '+' means "all available space". And vice-versa you can reduce the size of the partition; it's possioble to specify the size in absolute numbers (without +/- signs), or in sectors. The next example reduce size to 100MiB and move to old good offset 2048:
       
        echo "2048,100M" | ./sfdisk -N 1 /dev/sdc
        ...
        Old situation:
       
        Device     Start     End Sectors  Size Type
        /dev/sdc1   6144 2047966 2041823  997M Linux filesystem
       
        New situation:
 
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 206847  204800  100M Linux filesystem
Note that fdisks are too low-level to care about filesystems when resize, it's all about partition tables only.

Tuesday, April 14, 2015

persistent namespaces

Today I merged support for persistent namespaces to unshare(1). The persistent namespace does not require any running process with in the namespace and it's possible to enter the namespace by nsenter(1).

For example let's create a new UTS namespace and set a different hostname within the namespace:
  # hostname
  ws
  # touch /root/ns-uts         
  # unshare --uts=/root/ns-uts
  # hostname FooBar
  # exit
Now there is no process in the namespace, try to enter the namespace by --uts=/root/ns-uts reference:
  # nsenter --uts=/root/ns-uts
  # hostname
  FooBar 
  # exit
The reference to the namespace is bind mount to /proc/[pid]/ns/[type], so umount(8) is enough to remove the reference:
  # umount /root/ns-uts
If there is no another reference or any running process within the namespace then the namesapce is destoyed.

It's also possible to create another types of the persistent namespaces (--net, --ipc, ...). Don't forget that if you want to create a persistent mount namespace than the file (--mount=file) has to be on "private" filesystem, for example on Fedora where all is "shared" you have to use:

  # mount --bind /mnt/test /mnt/test
  # mount --make-rprivate /mnt/test
  # touch /mnt/test/my-ns
  # unshare --mount=/mnt/test/my-ns
  ...
Note that PID namespace cannot be without a running process (or more precisely the PID namespace is dead thing after init process (PID 1) termination).

Tuesday, March 31, 2015

cfdisk extra partition info

I have merged a new cfdisk(8) improvement. It allows to display/hide additional information about the current (selected) partition:
  • filesystem type, LABEL, UUID

    The important detail is that libblkid gathers the info by seeking on whole-disk device according to partition offsets from partition table -- so /dev/sda[n] devices do not have to exist or you can use cfdisk for disk images (e.g. "cfdisk file.img").
  • partition NAME, TYPE (hex/uuid and human readable name)
  • mountpoint -- for mounted filesystem as well as for not mounted (then from /etc/fstab)

All you need is press 'x'.  

The change is going to be available in util-linux v2.27 (May/Jun 2015). Thanks to Ondrej Oprala who has worked on this with me.

Friday, November 14, 2014

(s)fdisk scripts

Yesterday I merged support for sfdisk scripts to fdisk and cfdisk. Now it's possible to save your partitioning layout to text files and (re)use it in all fdisks.

It means that you don't have to start your partitioning from scratch in programs like cfdisk or fdisk, but you can reuse and finalize partitioning scenario already defined by sfdisk scripts. It may be attractive for DIY system installers, admins, etc.

All is based on libfdisk, so the same functionality may be very easily implemented in another tools too.

It's Friday ...sooo I tried to record video with some trivial example, all is from util-linux git tree (not yet release v2.26).


Monday, October 13, 2014

new sfdisk

The last partitioning tool from util-linux package without support for GPT (BSD, SGI, ...) and 4K disks was sfdisk. The situation was exactly the same like with cfdisk, it was impossible to incrementally improve the code, so the new version is written from scratch and based on libfdisk.

The most visible change is that the new sfdisk is all about sectors, it does not support obsolete Cylinder-Head-Sector addressing at all. This is no problem for users who use "--unit S", but if you still rely on old default cylinders than be careful with update. Frankly, LBA addressing is mandatory since ATA-3 (1997) and supported by Linux since beginning.

The new sfdisk also does not support some obscure or rarely used options like
--DOS, --IBM, --DOS-extended, --unhide, --show-extended, --cylinders, --heads, --sectors, --inside-outer, --not-inside-outer.

The last incompatibility is in backup mode. The old implementation uses sfdisk specific backup files. The new implementation uses the same concept like "wipefs --backup". The data are stored to ~/sfdisk-dev-offset.bak by default and it's possible to use dd(1) to restore your partition table or so. The backup files contains only data from the device, nothing else.  For example:
# sfdisk --backup /dev/sdb
...
Backup files:
        PMBR (offset     0, size   512): /root/sfdisk-sdb-0x00000000.bak
  GPT Header (offset   512, size   512): /root/sfdisk-sdb-0x00000200.bak
 GPT Entries (offset  1024, size 16384): /root/sfdisk-sdb-0x00000400.bak
...
and to restore:
 
 # dd if=~/sfdisk-sda-0x00000200.bak of=/dev/sdb \
      seek=$((0x00000200.bak)) bs=1 conv=notrunc 

All is described in the sfdisk man page. The another change is possibility to specify partition sizes in human readable notation {K,M,G,T...}, for example

# sfdisk /dev/sdb <<EOF
label: gpt
, 10G
, 10G
EOF

creates two 10GiB partitions. Note that default is to align all partitions to I/O limits (e.g. physical sector size). The first partition offset is by default 1MiB (e.g. 2048 512-byte-sectors). The partitions are by default aligned to megabytes. The same concept we use for fdisk and cfdisk (and parted probably too).

The default is DOS (MBR) disk label. If you want to use GPT then just add "--label gpt" to the command line or "label: gpt" to the script. The partition type shortcuts like 'L' (for Linux) or 'S' (for swap area) work for MBR as well as for GPT. If you want something else then you can use GUID for GPT or hex codes for MBR.

The new sfdisk supports nested disk labels. This is important for people who use BSD disk labels or hybrid GPT. The new command line option "--label-nested dos" forces sfdisk to modify protective MBR rather than the default GPT disk label. It means that you can manually create hybrid GPT. (No, we don't plan to add any "translate GPT to MBR" high-level feature for hybrid disks, hybrid GPT sucks.)

The nice new feature is that sfdisk allows to add new partitions to your partition table rather than always create whole partition table from scratch, all you need is the new --append command line option.

The important feature is that you can define your own output format for --list by new command line option "--output columns". sfdisk shares this new feature with fdisk where -o modifies 'p'rint output, for example:

 
# sfdisk --quiet --list -o DEVICE,SIZE,TYPE /dev/sda
Device      Size Type
/dev/sda1  1000M EFI System
/dev/sda2     2G Microsoft basic data
/dev/sda3   9.7G Linux swap
/dev/sda4  34.2G Microsoft basic data
/dev/sda5  63.2G Microsoft basic data
/dev/sda6  39.1G Microsoft basic data

You probably already know this concept from lsblk or findmnt. It's also supported to extend the default output by "-o+" notation, for example "fdisk --list -o+UUID /dev/sda" to see UUIDs for GPT partitions. 

Another features:
  • --part-uuid print or change GPT partition UUID
  • --part-type (or original --id) print or change partition type
  • --part-label print or changes GPT partition label (name)
  • --part-attrs print or change GPT partition attribute bites

The script parsing and --dump functionality is within libfdisk. The goal is to support partitioning dumps in fdisk and cfdisk too. It is going to be possible to "sfdisk --dump /dev/sda > foo" and then read the "foo" file to cfdisk or fdisk. And vice-versa, you can compose all your partition table by user-friendly cfdisk, save to the script file (rather than to device) and later use the file by sfdisk, etc.