Monday, October 13, 2014

new sfdisk

The last partitioning tool from util-linux package without support for GPT (BSD, SGI, ...) and 4K disks was sfdisk. The situation was exactly the same like with cfdisk, it was impossible to incrementally improve the code, so the new version is written from scratch and based on libfdisk.

The most visible change is that the new sfdisk is all about sectors, it does not support obsolete Cylinder-Head-Sector addressing at all. This is no problem for users who use "--unit S", but if you still rely on old default cylinders than be careful with update. Frankly, LBA addressing is mandatory since ATA-3 (1997) and supported by Linux since beginning.

The new sfdisk also does not support some obscure or rarely used options like
--DOS, --IBM, --DOS-extended, --unhide, --show-extended, --cylinders, --heads, --sectors, --inside-outer, --not-inside-outer.

The last incompatibility is in backup mode. The old implementation uses sfdisk specific backup files. The new implementation uses the same concept like "wipefs --backup". The data are stored to ~/sfdisk-dev-offset.bak by default and it's possible to use dd(1) to restore your partition table or so. The backup files contains only data from the device, nothing else.  For example:
# sfdisk --backup /dev/sdb
Backup files:
        PMBR (offset     0, size   512): /root/sfdisk-sdb-0x00000000.bak
  GPT Header (offset   512, size   512): /root/sfdisk-sdb-0x00000200.bak
 GPT Entries (offset  1024, size 16384): /root/sfdisk-sdb-0x00000400.bak
and to restore:
 # dd if=~/sfdisk-sda-0x00000200.bak of=/dev/sdb \
      seek=$((0x00000200.bak)) bs=1 conv=notrunc 

All is described in the sfdisk man page. The another change is possibility to specify partition sizes in human readable notation {K,M,G,T...}, for example

# sfdisk /dev/sdb <<EOF
label: gpt
, 10G
, 10G

creates two 10GiB partitions. Note that default is to align all partitions to I/O limits (e.g. physical sector size). The first partition offset is by default 1MiB (e.g. 2048 512-byte-sectors). The partitions are by default aligned to megabytes. The same concept we use for fdisk and cfdisk (and parted probably too).

The default is DOS (MBR) disk label. If you want to use GPT then just add "--label gpt" to the command line or "label: gpt" to the script. The partition type shortcuts like 'L' (for Linux) or 'S' (for swap area) work for MBR as well as for GPT. If you want something else then you can use GUID for GPT or hex codes for MBR.

The new sfdisk supports nested disk labels. This is important for people who use BSD disk labels or hybrid GPT. The new command line option "--label-nested dos" forces sfdisk to modify protective MBR rather than the default GPT disk label. It means that you can manually create hybrid GPT. (No, we don't plan to add any "translate GPT to MBR" high-level feature for hybrid disks, hybrid GPT sucks.)

The nice new feature is that sfdisk allows to add new partitions to your partition table rather than always create whole partition table from scratch, all you need is the new --append command line option.

The important feature is that you can define your own output format for --list by new command line option "--output columns". sfdisk shares this new feature with fdisk where -o modifies 'p'rint output, for example:

# sfdisk --quiet --list -o DEVICE,SIZE,TYPE /dev/sda
Device      Size Type
/dev/sda1  1000M EFI System
/dev/sda2     2G Microsoft basic data
/dev/sda3   9.7G Linux swap
/dev/sda4  34.2G Microsoft basic data
/dev/sda5  63.2G Microsoft basic data
/dev/sda6  39.1G Microsoft basic data

You probably already know this concept from lsblk or findmnt. It's also supported to extend the default output by "-o+" notation, for example "fdisk --list -o+UUID /dev/sda" to see UUIDs for GPT partitions. 

Another features:
  • --part-uuid print or change GPT partition UUID
  • --part-type (or original --id) print or change partition type
  • --part-label print or changes GPT partition label (name)
  • --part-attrs print or change GPT partition attribute bites

The script parsing and --dump functionality is within libfdisk. The goal is to support partitioning dumps in fdisk and cfdisk too. It is going to be possible to "sfdisk --dump /dev/sda > foo" and then read the "foo" file to cfdisk or fdisk. And vice-versa, you can compose all your partition table by user-friendly cfdisk, save to the script file (rather than to device) and later use the file by sfdisk, etc.

Wednesday, August 6, 2014


I merged zramctl tool a few days ago. It's based on original code from Timofey Titovets and the idea is the same as for losetup(8). zRam was merged to Linux kernel 3.14; the zram devices use RAM to create compressed block devices. The zramctl provides user friendly command line interface to create, reset and list the devices.

It's pretty simple and easy to use from userspace. See example below (kernel 3.15.7-200.fc20)
# modprobe zram num_devices=4
# zramctl --find --size 10M
# mkfs.ext2 -q /dev/zram0
# mount /dev/zram0 /mnt/test
./zramctl --find --size 10M --streams 2
# mkswap /dev/zram1
Setting up swapspace version 1, size = 10236 KiB
no label, UUID=776a95b7-5876-4be4-a66b-9b869cf35bd1
# swapon UUID=776a95b7-5876-4be4-a66b-9b869cf35bd1
# zramctl
/dev/zram0 lzo            10M  40K  797B   40K       1 /mnt/test
/dev/zram1 lzo            10M   4K   78B    4K       2 [SWAP]

Note that kernel 3.14 does not support more compression streams and algorithms selection.

The next step will be probably to support zram initialization in swapon(8) to
make it easy for users to define swap on zram in /etc/fstab.

Friday, June 20, 2014

new cfdisk (util-linux v2.25)

cfdisk is a popular ncurses partitioning tool. Unfortunately, it has been ignored by developers for many many years. The original version does not a have clue
about 4K disks, modern disk I/O limits (alignment offset, optimal I/O sizes, etc),
and it was all about MBR (DOS) only.

It was impossible to improve the old code by refactoring, so the new version is written from scratch. The new cfdisk is based on on the same code like fdisk, so it shares all the necessary features required for new disks and it supports MBR, GPT, SGI and SUN disk labels. Some features have been removed:
  • it does not have a clue about CHS addressing any more
  • it's impossible to play any games with disk geometry (use fdisk if you believe that CHS setting makes any sense)
  • 'm' command (maximize disk usage of the current partition) is not implemented
  • 'p' command and --print command line option are not implemented (use partx or lsblk)
  •  'u' (units) command is not implemented, all is based on sectors or real sizes
The idea is to keep cfdisk pretty simple and easy to use for end users. Advanced users or users who want to do some unusual things have to use fdisk or parted.

New features:
  • output formatting based on libsmartcols
  • tree-like output for MBR extended and logical partitions
  • show free space on place where the gap is really located
  • colors
  • new 's'ort command to reorder partition according to start sectors
  • partitions aligned to I/O limits, 1MiB start offset, etc.
  • it's still possible to recompile with slang library (although I personallyprefer ncursesw)

For full size screenshots see g+

What next? sfdisk and stable public API fro libfdisk... let's wait for v2.26 :-)

Friday, May 30, 2014

libsmartcols - pretty output for everyone!

util-linux 2.25 is going to contain libsmartcols. The library code is based on code originally developed for lsblk(8), findmnt(8), lslocks(8) etc.  The goal is to share formatting code to print pretty tables and trees.

  • tree-like output (see lsblk(8) or findmnt(8))
  • table-like output (see lslocks(8))
  • formatting sensitive to terminal width
  • fully supports UTF8 (correctly truncate data in cells, tree lines, ...)
  • dynamic or fixed columns width
  • truncate data in cell on demand
  • raw (no formatting) output mode
  • NAME=value output mode
  • custom cells and lines delimiters (for example for CSV output)
  • built-in ASCII, UTF8 or custom symbols to draw trees
  • per column, line or cell colors
  • colors specified by words (e.g. "red") or ESC sequences
  • sort lines
  • enable/disable heading
  • output to FILE stream (default stdout), or to string
  • all columns, lines and cells accessible by iterators (no callbacks)
  • output data specified by strings (no crazy unions or to-string conversion functions in the library)
  • reference counting
  • integrated debug output (sensitive to LIBSMARTCOLS_DEBUG env)
  • usable to keep parent<->child relationship
  • library symbols versioning (no soname changes)

See bellow a trivial (so no errors handling, etc.) example.

int main(void)
        struct libscols_table *tb;
        struct libscols_line *ln, *dad, *gdad;
        enum { COL_NAME, COL_AGE };

        setlocale(LC_ALL, "");

        tb = scols_new_table();
        scols_table_new_column(tb, "NAME", 0.1, SCOLS_FL_TREE);
        scols_table_new_column(tb, "AGE",     2, SCOLS_FL_RIGHT);

        ln = gdad = scols_table_new_line(tb, NULL);
        scols_line_set_data(ln, COL_NAME, "Grandfather Bob");
        scols_line_set_data(ln, COL_AGE, "61");

        ln = dad = scols_table_new_line(tb, ln);
        scols_line_set_data(ln, COL_NAME, "Father Adam");
        scols_line_set_data(ln, COL_AGE, "38");

        ln = scols_table_new_line(tb, dad);
        scols_line_set_data(ln, COL_NAME, "Baby Val");
        scols_line_set_data(ln, COL_AGE, "9");

        ln = scols_table_new_line(tb, dad);
        scols_line_set_data(ln, COL_NAME, "Baby Dilbert");
        scols_line_set_data(ln, COL_AGE, "5");

        ln = scols_table_new_line(tb, gdad);
        scols_line_set_data(ln, COL_NAME, "Aunt Gaga");
        scols_line_set_data(ln, COL_AGE, "35");

        return 0;

... and output:

NAME             AGE
Grandfather Bob   61
├─Father Adam     38
│ ├─Baby Val       9
│ └─Baby Dilbert   5
└─Aunt Gaga       35 

Tuesday, April 1, 2014


I have already talked about it at g+. For util-linux we don't want to follow the current strange behavior when command-line utils have disabled colors by default, but distributions use shell aliases in profile files to revert the default.

It does not mean that we want ignore color blind people, obscure terminals or people who just don't like colors. The solution is to provide simple, elegant and package independent way how to control (disable) colorized output. And yes, aliases are not elegant.

The next util-linux release is going to introduce /etc/terminal-colors.d directory
to control colors for terminal utils. It's pretty simple:

  •  disable colors for all utils:
    # touch /etc/terminal-colors.d/disable 
  • disable colors for dmesg(1):
    # touch /etc/terminal-colors.d/dmesg.disable
  • disable colors for all utils except dmesg(1):
    # touch /etc/terminal-colors.d/disable
    # touch /etc/terminal-colors.d/dmesg.enable
For utils from util-linux the command line option --color[={auto,never,always}] overrides the setting from terminal-colors.d (so if you for some reason love shell aliases then it's still possible...).

The terminal-colors.d concept is completely open, it would be nice to see it supported in another projects (hint: gcc, grep, coreutils:-) too. It's so trivial that all you need is a few access(F_OK) calls. The thing we maintain in util-linux is terminal-colors.d.5 man page only.

Maybe later for util-linux we will also use the directory (or/and copy in $HOME) to store color schemes.

Keep your Bike-sheds colorized! :-)

UPDATE: since v2.25:
  • $XDG_CONFIG_HOME/terminal-colors.d  or  $HOME/.config/terminal-colors.d overrides the global setting
  • terminal-colors.d is possible to use for color schemes
        echo 'alert 37;41' >> /etc/terminal-colors.d/dmesg.scheme
    changes color used for kernel alerts in dmesg output
  • synopsis is /etc/terminal-colors.d/[[name][@term].][type], it means that for example /etc/terminal-colors.d/@vt100.disable disables colors for all vt100 terminals
  • we have terminal-colors.d(5) man page with all details

Wednesday, February 5, 2014

stable mutt-kz

The first official mutt-kz release with tarballs is available now. If you have no idea about mutt-kz: it is mutt fork with virtual folders, tags and fulltext search (based on notmuch), supports INBOX->archive use case, integrated index-color and sidebar patches. All changes to the original mutt are backported to mutt-kz.

I'm really happy that there is a community around the project with testers and contributors. The mutt-kz mailing list has 80 members now. Thanks!


  • virtual folder (based on notmuch queries) - defined in .muttrc
  • virtual folders on-the-fly generated from queries
  • command to see entire thread for specific message (including already archived messages or messages excluded from the current virtual folder)
  • UI to modify notmuch labels 
  • labels transformation to special chars (e.g. funny unicode text icons) in mutt index
  • new %g and %Gx mutt index and pager format sequences to display notmuch labels 
  • special labels for sent emails 
  • really smart fulltext search (thanks to notmuch and Xapian), for example: "from:kzak tag:ul date:1year.." to search for my posts to util-linux (ul tag) mailing list in last year. 
  • nested if-else conditional and date format expressions (for index format)
  • new mutt pattern modifier for labels 
  • all implemented by regular mutt commands (no hacks) so usable in mutt hooks and macros 
  • you really don't have to delete messages, just move things to archive, notmuch is fast enough to work with large sets of messages 
  • sidebar to show mailboxes or virtual folders 
  • and more... 
my .muttrc:

Tuesday, October 15, 2013

util-linux v2.24 - fdisk(8)

This release is mostly about small incremental changes, a small exception is fdisk(8).

I have finished fdisk(8) refactoring, this task we started with Davidlohr Bueso and Petr Uzel two releases ago. The result is a code that is easy to extend and all disklabel specific code is no more maintained together as a ball of spaghetti.

The goal is to have one library (libfdisk) shared between all the fdisks (sfdisk, cfdisk, ...). Now it's used in fdisk only. I hope that one day we're going to have shared library with a public stable API.

 The most visible changes:
  • fdisk dialogs and output unification, more verbose messages
  • colors for warnings
  • new (but backwardly compatible) menus
  • all operations for all disklabels based on sectors by default
  • man page clean up
  • improved GPT disklabel support (rename partition, partition and disk UUID modification)

    The last missing thing is support for manual modification of hybrid GPT. This is planned for the next release v2.25.

    My laptop with GPT:
# fdisk -l /dev/sda

Disk /dev/sda: 149.1 GiB, 160041885696 bytes, 312581808 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 3549290F-417C-4941-8503-F7835109B821

Device           Start          End   Size Type
/dev/sda1         2048      2050047  1000M EFI System
/dev/sda2      2050048      6146047     2G Microsoft basic data
/dev/sda3      6146048     26462207   9.7G Linux swap
/dev/sda4     26462208     98142207  34.2G Microsoft basic data
/dev/sda5     98142208    230662143  63.2G Microsoft basic data
/dev/sda6    230662144    312580095  39.1G Microsoft basic data
  • disklabel hex dump (command 'D')

    This is nice feature for advanced users. You don't have to fight with hexdump, search the right offsets etc. fdisk(8) will locate all relevant disklabel parts on the device and print all in hex format. (Don't forget that many things don't use static offsets -- for example GPT array with partitions, MBR extended partitions, etc).

  • For example (disk with two logical partitoins):
# echo -e 'x\nD\nq\n' | fdisk  /dev/sdb
Welcome to fdisk (util-linux 2.24.rc2-22-1d8c-dirty).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): 
Expert command (m for help): 
MBR: offset = 0, size = 512 bytes. 
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
000001b0  00 00 00 00 00 00 00 00  14 92 2b 9e 00 00 00 20
000001c0  21 00 83 df 13 0c 00 08  00 00 00 20 03 00 00 df
000001d0  14 0c 05 7c 30 4c 00 28  03 00 00 98 0f 00 00 00
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa
EBR: offset = 105906176, size = 512 bytes.
06500000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
065001c0  34 0d 83 bf 26 19 00 08  00 00 00 20 03 00 00 bf
065001d0  27 19 05 9f 39 26 00 28  03 00 00 28 03 00 00 00
065001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
065001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa
EBR: offset = 211812352, size = 512 bytes.
0ca00000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
0ca001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 e0
0ca001c0  08 19 83 9f 39 26 00 08  00 00 00 20 03 00 00 00
0ca001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
0ca001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa