Lightweight Clones “cp –reflink”
posted on November 7, 2016 by Amit Golander
Cloning a file, also commonly referred to as creating a Snapshot, is a very useful operation. Especially, if you can clone the data in a rapid, simple, low-overhead and coherent manner.
Cloning usage varies. Creating on-demand database images for backups; replicating machine images (e.g., VDI); and forking live production environments in order to provide an instantly-available safe sandbox for developers, data scientists or QA, are just a few usage examples. There are even situations where cloning can be integrated into analytical processes, generating multiple data images for simulations employing Monte Carlo methods.
Clones have been proven to be a popular storage feature. The capability, at different implementation qualities, is offered by many storage vendors. A good Lightweight Clone implementation has four desired properties.
- Support for a high number of clones. Early implementations had an upper bound of few read-only snapshots of the entire volume. The desired solution however is to have thousands of concurrent clones, all with their own read/write permissions at a file-level granularity.
- Lightweight Clones should be efficient in Time and Space. Clones are expected to be rapid and coherent, so that it does not slow down regular access to the data. They are also expected to be efficient in capacity utilization. There is no reason for similar clones to store the identical portion of the data twice.
- File-level clones are easier to use. Users and application administrators understand files and have access to file-level operations, whereas full volume operations require administrator involvement and are thus less useful.
- Last, but not least, a standard API is highly desirable. No one wants to spend time writing code for a specific vendor, digging a lock-in grave. Luckily a new standard way was recently introduced in the form of a POSIX API enhancement:
cp –reflink SrcFile CopyFile
New POSIX API for Lightweight Clones
Consider the following user space code:
# cp dir/SrcFile dir/CopyFile
The traditional copy operation reads the SrcFile data to a user space buffer and writes it back to CopyFile.
Post the copy command:
# ls dir/* → Reveals two unrelated files SrcFile and CopyFile.
Each could have its own permissions, timestamps etc.
# du -sh dir → Ignoring other files, the disk usage output would be twice
the capacity of SrcFile, because its data was duplicated.
Now let’s view the same set of commands, but this time using the newly
standardized reflink flag, the lightweight clone version:
# cp –reflink dir/SrcFile dir/CopyFile
# ls dir/* → It still shows two unrelated files
# du -sh dir → but… there is no waste. Data is shared across files
Another flag option to consider using is the auto option, so that if someone ever places the files on a file system that does not support lightweight clones – your code will still run, even if inefficiently, by falling back to standard copy.
# cp —reflink=auto dir/SrcFile dir/CopyFile
Additional options are depicted on the cp man page.
Native support in Plexistor SDM v2.0
The new standard POSIX API for lightweight clones is supported in the latest 2.0 release of Plexistor SDM. The implementation supports thousands of coherent file-level clones per file. A lightweight clone command completes with almost zero impact on latency and capacity.
Plexistor SDM is also the first solution to support lightweight clones alongside the direct PM-access (DAX) feature. Other storage solutions that support DAX treat it as second class citizen and disable most other features when DAX is activated. Plexistor believes that users should be free to access their data using any access method or pattern they decide on, at run time and without giving up on enterprise-grade data services.
Now that lightweight file-level cloning has been made publically available in Plexistor SDM 2.0, we look forward to hearing from the community of users about their experiences with it.
Are you doing something new with cloning, something you couldn’t even consider attempting before because you couldn’t find the combination of ultra-low latency, data persistence, and lightweight cloning to meet your needs? Please let us know!