Thursday, July 23, 2015

lsipc new command to list IPC facilities

Although it seems that System V IPC is obsolete thing, it's still used by many (enterprise) tools like databases, application servers etc.

The classic tool ipcs(1) provides basic overview, unfortunately the output is very fixed and it's impossible to mix the output columns (for example --pid and --creator cannot be used together etc.). It's also very difficult to extend the current implementation without break backward compatibility.

The solution is a new libsmartcols based tool: lsipc(1). It's like many another util-linux ls-like tools, columns are maintains independently, it's easy to extend, user has absolute control on the output and it provides more output formats (including JSON).

The important and unique is the default (--global) output:
# lsipc
RESOURCE DESCRIPTION                            LIMIT   USED  USE%
MSGMNI   Number of message queues               32000      0 0.00%
MSGMAX   Max size of message (bytes)             8192      0 0.00%
MSGMNB   Default max size of queue (bytes)      16384      0 0.00%
SHMMNI   Shared memory segments                  4096     21 0.51%
SHMALL   Shared memory pages                268435456 294049 0.11%
SEMMNS   Total number of semaphores        1024000000      0 0.00%
SEMMNI   Number of Semaphore IDs                32000      0 0.00%
the information about shared memory is provided with COMMAND column, that makes things more human friendly:
# lsipc --shmem
KEY        ID           PERMS OWNER SIZE NATTCH STATUS CTIME  CPID LPID COMMAND
0x6c6c6536 0        rw-------  root   4K      0        Jul16   288  288
0x00000000 327681   rw-------  kzak   4M      2 dest   Jul16  1487  992 gjs /home/kzak/.local/share/gnome-shell/extensions/
0x00000000 393219   rw-------  kzak 512K      2 dest   Jul16  1141 2853 /usr/bin/gnome-shell
0x00000000 2064389  rw-------  kzak 384K      2 dest   Jul17  9443  992 xchat
0x00000000 47611910 rw-------  kzak   8M      2 dest   10:07  2853  804 /usr/lib64/firefox/firefox
0x00000000 2228231  rw-------  kzak 384K      2 dest   Jul17  9443  992 xchat
0x00000000 2195464  rw-------  kzak   2M      2 dest   Jul17  9443  992 xchat
0x00000000 42303503 rw-------  kzak   8M      2 dest   Jul22  1340  992 /usr/bin/gnome-software --gapplication-service
0x00000000 42270736 rw-------  kzak   4M      2 dest   Jul22  1340  992 /usr/bin/gnome-software --gapplication-service
0x00000000 43188243 rw-------  kzak   4M      2 dest   Jul22  8873 1845 /usr/libexec/gnome-terminal-server
0x00000000 33882133 rw-------  kzak 384K      2 dest   Jul20 26049  992 /usr/bin/python3 /usr/bin/hp-systray --force-startup
and you can define your own output:
# lsipc --shmem -o KEY,SIZE,PERMS
KEY        SIZE     PERMS
0x6c6c6536   4K rw-------
0x00000000   4M rw-------
0x00000000 512K rw-------
0x00000000 384K rw-------
0x00000000   8M rw-------
0x00000000 384K rw-------
0x00000000   2M rw-------
0x00000000   8M rw-------
0x00000000   4M rw-------
0x00000000   4M rw-------
0x00000000 384K rw-------
or ask for specific segment:
# lsipc --shmem --id 47611910
Key:                                0x00000000
ID:                                 47611910
Owner:                              kzak
Permissions:                        rw-------
Creator UID:                        1000
Creator user:                       kzak
Creator GID:                        1000
Creator group:                      kzak
UID:                                1000
User name:                          kzak
GID:                                1000
Group name:                         kzak
Last change:                        Thu Jul 23 10:07:46 2015
Segment size:                       8M
Attached processes:                 2
Status:                             dest
Attach time:                        Thu Jul 23 10:07:46 2015
Creator command:                    /usr/lib64/firefox/firefox
Creator PID:                        2853
Last user PID:                      804
and it should be easy to export complex info about IPC to the monitoring tool:
# lsipc --json -o resource,limit,used
{
   "ipclimits": [
      {"resource": "MSGMNI", "limit": "32000", "used": "0"},
      {"resource": "MSGMAX", "limit": "8192", "used": "0"},
      {"resource": "MSGMNB", "limit": "16384", "used": "0"},
      {"resource": "SHMMNI", "limit": "4096", "used": "11"},
      {"resource": "SHMALL", "limit": "268435456", "used": "8097"},
      {"resource": "SEMMNS", "limit": "1024000000", "used": "0"},
      {"resource": "SEMMNI", "limit": "32000", "used": "0"}
   ]
}
lsipc(1) will be available in util-linux v2.27 (probaly August 2015).

Monday, June 15, 2015

sfdisk --json

As promised by the previous blog I have merged support for JSON dump to sfdisk/libfdisk.
# sfdisk --json /dev/sda
{
   "partitiontable": {
      "label": "gpt",
      "id": "E471810B-5954-48E6-B4F0-5D369ADCC514",
      "device": "/dev/sda",
      "unit": "sectors",
      "firstlba": 34,
      "lastlba": 468862094,
      "partitions": [
         {"node": "/dev/sda1", "start": 2048, "size": 409600, "type": "C12A7328-F81F-11D2-BA4B-00A0C93EC93B", "uuid": "2062335E-51B6-49E8-BC3C-4C6D449E6805", "name": "EFI System Partition"},
         {"node": "/dev/sda2", "start": 411648, "size": 409600, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "DF378D5F-B7BA-4887-BAEB-A81E48DE903D"},
         {"node": "/dev/sda3", "start": 821248, "size": 273266688, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "6073277F-87BC-43FF-BCFD-724C4484A63A"},
         {"node": "/dev/sda4", "start": 274087936, "size": 104857600, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "EC5A52E0-FE99-4B38-91E7-B5FF1120E0E5"},
         {"node": "/dev/sda5", "start": 378945536, "size": 73531392, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "11A62EAE-5ECA-4EF5-8BF5-DD18A4115F4A"},
         {"node": "/dev/sda6", "start": 452476928, "size": 16384000, "type": "0657FD6D-A4AB-43C4-84E5-0933C84B4F4F", "uuid": "A13DACFE-6FFB-4B2C-BD89-B91B768A4F77"}
      ]
   }
}

Friday, June 5, 2015

JSON output for basic tools

I have merged into libsmartcols support for JSON format. The feature is currently enabled for lsblk(8), findmnt(8), lslocks(8) and losetup(8). The library already uses structured data, so add a different output format is pretty trivial. For example findmnt(8) and lsblk(8) are able to gather much information and standardized parsable format makes it easy to use it monitoring tools, automatic bug reporting tools, etc.
$ lsblk --json -o name,type,mountpoint                                    
{                                                                         
   "blockdevices": [                                                      
      {"name": "sda", "type": "disk", "mountpoint": null,                 
         "children": [                                                    
            {"name": "sda1", "type": "part", "mountpoint": "/boot/efi"},  
            {"name": "sda2", "type": "part", "mountpoint": "/boot"},      
            {"name": "sda3", "type": "part", "mountpoint": "/home"},      
            {"name": "sda4", "type": "part", "mountpoint": "/"},          
            {"name": "sda5", "type": "part", "mountpoint": "/home/wine"}, 
            {"name": "sda6", "type": "part", "mountpoint": "[SWAP]"}      
         ]                                                                
      },                                                                  
      {"name": "sdb", "type": "disk", "mountpoint": null,                 
         "children": [                                                    
            {"name": "sdb1", "type": "part", "mountpoint": "/home/archive"}
         ]                                                                
      }                                                                   
   ]                                                                      
}                                                                         
                                                                    
$ findmnt --json --submounts /home                                        
{                                                                         
   "filesystems": [                                                       
      {"target": "/home", "source": "/dev/sda3", "fstype": "ext4", "options": "rw,relatime,data=ordered",
         "children": [                                                    
            {"target": "/home/wine", "source": "/dev/sda5", "fstype": "ext4", "options": "rw,relatime,data=ordered"},
            {"target": "/home/archive", "source": "/dev/sdb1", "fstype": "ext4", "options": "rw,relatime,data=ordered"}
         ]                                                                
      }                                                                   
   ]                                                                      
}
For the next Monday my TODO contains "JSON sfdisk/libfdisk dump output"...

Wednesday, May 6, 2015

resize by sfdisk

The last week released util-linux v2.26.2 contains improved (fixed:-) sfdisk. The most visible change is that the new version supports partition resize and start offset move.

For example 1GiB disk with GPT:

        echo ", +10M" | ./sfdisk -N 1 /dev/sdc
        ...
       
        Old situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 206847  204800  100M Linux filesystem
       
        New situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 227327  225280  110M Linux filesystem
the first partition (-N 1) has been enlarged from 100MiB to 110MiB.

You can also move begin of the partition:

        echo "+2M" | ./sfdisk -N 1 /dev/sdc
        ...
        Old situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 227327  225280  110M Linux filesystem
       
        New situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   6144 231423  225280  110M Linux filesystem
and it's possible to enalarge as much as possible:
       
        echo ", +" | ./sfdisk -N 1 /dev/sdc
        ...
        Old situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   6144 231423  225280  110M Linux filesystem
       
        New situation:
 
        Device     Start     End Sectors  Size Type
        /dev/sdc1   6144 2047966 2041823  997M Linux filesystem
the '+' means "all available space". And vice-versa you can reduce the size of the partition; it's possioble to specify the size in absolute numbers (without +/- signs), or in sectors. The next example reduce size to 100MiB and move to old good offset 2048:
       
        echo "2048,100M" | ./sfdisk -N 1 /dev/sdc
        ...
        Old situation:
       
        Device     Start     End Sectors  Size Type
        /dev/sdc1   6144 2047966 2041823  997M Linux filesystem
       
        New situation:
 
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 206847  204800  100M Linux filesystem
Note that fdisks are too low-level to care about filesystems when resize, it's all about partition tables only.

Tuesday, April 14, 2015

persistent namespaces

Today I merged support for persistent namespaces to unshare(1). The persistent namespace does not require any running process with in the namespace and it's possible to enter the namespace by nsenter(1).

For example let's create a new UTS namespace and set a different hostname within the namespace:
  # hostname
  ws
  # touch /root/ns-uts         
  # unshare --uts=/root/ns-uts
  # hostname FooBar
  # exit
Now there is no process in the namespace, try to enter the namespace by --uts=/root/ns-uts reference:
  # nsenter --uts=/root/ns-uts
  # hostname
  FooBar 
  # exit
The reference to the namespace is bind mount to /proc/[pid]/ns/[type], so umount(8) is enough to remove the reference:
  # umount /root/ns-uts
If there is no another reference or any running process within the namespace then the namesapce is destoyed.

It's also possible to create another types of the persistent namespaces (--net, --ipc, ...). Don't forget that if you want to create a persistent mount namespace than the file (--mount=file) has to be on "private" filesystem, for example on Fedora where all is "shared" you have to use:

  # mount --bind /mnt/test /mnt/test
  # mount --make-rprivate /mnt/test
  # touch /mnt/test/my-ns
  # unshare --mount=/mnt/test/my-ns
  ...
Note that PID namespace cannot be without a running process (or more precisely the PID namespace is dead thing after init process (PID 1) termination).

Tuesday, March 31, 2015

cfdisk extra partition info

I have merged a new cfdisk(8) improvement. It allows to display/hide additional information about the current (selected) partition:
  • filesystem type, LABEL, UUID

    The important detail is that libblkid gathers the info by seeking on whole-disk device according to partition offsets from partition table -- so /dev/sda[n] devices do not have to exist or you can use cfdisk for disk images (e.g. "cfdisk file.img").
  • partition NAME, TYPE (hex/uuid and human readable name)
  • mountpoint -- for mounted filesystem as well as for not mounted (then from /etc/fstab)

All you need is press 'x'.  

The change is going to be available in util-linux v2.27 (May/Jun 2015). Thanks to Ondrej Oprala who has worked on this with me.

Friday, November 14, 2014

(s)fdisk scripts

Yesterday I merged support for sfdisk scripts to fdisk and cfdisk. Now it's possible to save your partitioning layout to text files and (re)use it in all fdisks.

It means that you don't have to start your partitioning from scratch in programs like cfdisk or fdisk, but you can reuse and finalize partitioning scenario already defined by sfdisk scripts. It may be attractive for DIY system installers, admins, etc.

All is based on libfdisk, so the same functionality may be very easily implemented in another tools too.

It's Friday ...sooo I tried to record video with some trivial example, all is from util-linux git tree (not yet release v2.26).