3

We have a situation where an enormous set of template files (several tens of gigabytes) is required by multiple pieces of software, from multiple vendors. The applications want to open these template files the same way they opens any project file, i.e. read-write.

  • It's a burden on the network infrastructure to have users copy the template hive locally to their Linux workstations (RHEL6 and RHEL7 only)
  • The software does not make changes to the template files
  • If the files are write-protected on the server, the software throws an error
  • If the NFS volume is mounted read-only and the files are given write permission, the software throws an error
  • If we give write permissions on files on a writable NFS volume we can no longer trust the template files to be pristine

Short of eating that infrastructure burden, I'm looking for a way to fake out the software by making the NFS volume appear writable, giving the files write permissions, but not actually committing the changes.

I'm reminded of "journal rewind" tools like Deepfreeze on Windows NTFS volumes. Is there a similar hack for NFS mounts? When giving this question tags, "unionfs" popped up as a match for "nfs". That's also a method for allowing "fake writes" to Live CDs. I'm heading that direction now, but I hope someone's already done this.

Another constraint is that users are often required to use legacy versions of the software, from multiple vendors; which means a bugfix from the vendors is only going to help the newest version(s).

Rich
  • 135
  • 2
    Have you tried making the files immutable? – TheFiddlerWins Apr 06 '18 at 17:29
  • The volume is hosted on what ? – yagmoth555 Apr 06 '18 at 17:33
  • 1
    Have you filed a bug report with the software vendor? – Michael Hampton Apr 06 '18 at 18:20
  • @TheFiddlerWins I'll see about making files immutable. That sounds promising. – Rich Apr 06 '18 at 18:32
  • @yagmoth555 NetApp – Rich Apr 06 '18 at 18:33
  • @MichaelHampton Multiple vendors, all seemingly baffled that their customers would want to save network bandwidth. – Rich Apr 06 '18 at 18:33
  • 2
    Hm, maybe you could pull some tricks with an overlay on a tmpfs or something. – Michael Hampton Apr 06 '18 at 18:36
  • 1
    @Rich if It's a netapp you could migrate this portion of the file system to it's own volume/qtree/etc and then just restore it every night/day/hour etc – TheFiddlerWins Apr 06 '18 at 19:52
  • Found this question, basically the same thing. (Of course, live CDs...) https://unix.stackexchange.com/questions/181421/building-a-read-only-linux-system-with-a-writable-layer-in-ram -- one of the comments on a question states that unionfs isn't available in RHEL6/7, which is what I'm restricted to here. (Adding that fact to the question...) – Rich Apr 06 '18 at 21:37
  • I think our best solution will be to create some kind of "fill my disk" thing to copy the pristine copy down to the client overnight. We're talking 100+ workstations with at least semiannual updates from the vendors; plus a need to keep legacy versions alive. Still would not mind a way to solve this elegantly. – Rich Apr 06 '18 at 21:41
  • @TheFiddlerWins I'm asking around to see how that can be rolled into our processes. Sounds pretty good. – Rich Apr 06 '18 at 21:45
  • @TheFiddlerWins immutability also returns a failure code. I'm looking for a silent way to direct any write activity to /dev/null – Rich Apr 06 '18 at 21:47
  • @Rich BitTorrent might be the most efficient way to distribute the file to the workstations. As an added benefit it can verify if a previously downloaded file has been modified and only download the parts that have been modified. – kasperd Apr 08 '18 at 22:14
  • @kasperd At first blush, BitTorrent seems to be an excellent suggestion - it'd definitely save bandwidth. I'm not sure it would fly in our organization due its prevalence in illicit software distribution, but in general, on a private network, this is a sound proposition. Thanks! – Rich Apr 09 '18 at 18:40
  • 1
    @Rich By that reasoning your organization should avoid using computers altogether. – kasperd Apr 09 '18 at 20:12
  • @kasperd You have no idea... – Rich Apr 09 '18 at 21:59
  • I would never do this in a production environment, but there is also a fuse fs called nullfs that would discard everything. Here is a version that probably won't crash but you have to use bsd. – Aaron Apr 11 '18 at 21:47

5 Answers5

3

Best way is to ask the vendor to fix the software like others said.

Also check the license. Some software do not like being run shared on a file server (including sharing data on the file server). If that's the case, you may not be allowed to run it that way. Then just duplicate all the template files on each user's local disk. Disks are cheap, unless you are using SSDs.

If it's a software no longer supported and is within legal bounds in your country, you could hack the software and change read-write open mode to read-only open mode. Some disassembly required.

You could create a custom user-space virtual filesystem like vfs or fuse that fakes read-write mode, which may require some programming.

Yusuf N
  • 141
  • The software is being used to design a product. Hacking legacy versions of currently licensed software just is not going to fly. – Rich Apr 06 '18 at 22:14
  • "you may not be allowed to run it that way" -- that's certainly true for some software, but not the ones in question. – Rich Apr 06 '18 at 22:15
  • Roll-yer-own filesystem? Sure. In my copious spare time! lol I'm intrigued enough that I would love to give this a go; but I'll have to wait until I retire, kids out of the house, all that stuff. – Rich Apr 06 '18 at 22:17
  • Your only realistic suggestion is the first: ask the vendor for a fix. I wasn't terribly clear, but we thunk of that already, they're flummoxed, and the legacy versions would never get fixed. This is a good general piece of advice, though, so I voted you up. – Rich Apr 06 '18 at 22:18
3

Did you try overlayfs . It can fake writes and keep all changes, if any, locally.

kofemann
  • 4,866
  • I want to vote this one up and choose it as the accepted answer, since it's the best general solution; but RHEL is the OS that's typically chosen for this type of environment, where overlayfs is not an option. – Rich Apr 09 '18 at 22:31
  • 1
    That's right, it's included into RHEL7, but not into RHEL6. – kofemann Apr 10 '18 at 07:22
  • I didn't realize it had to be enabled in RHEL7.x. Correction: it's only supported since RHEL7.4; plus "* Only XFS is currently supported for use as a lower layer file system." Contradicted by the kernel doc file, which says "The lower filesystem can be any filesystem supported by Linux and does not need to be writable. The lower filesystem can even be another overlayfs." Still, it's a good general solution. – Rich Apr 10 '18 at 19:52
1

Since it's a netapp you could create a (thin) clone of the volume & share that to clients. It could be readable but deleted nightly & recreated. This is a little different than restoring from a snapshot like I said in the comments since the original FS can be RO (or just not shared) so you never risk accidentally altering the contents.

Otherwise, as others have said, zsync to local storage may be worth investigating.

TheFiddlerWins
  • 2,999
  • This is a clever workaround that saves network bandwidth and does not rely on special configuration of the client systems. – Rich Apr 17 '18 at 17:25
0

Overnight copy

Although this is the very thing we do not want, our best solution may be to create some kind of "fill my disk" thing to copy the pristine copy down to the client overnight. (We're talking 100+ workstations with at least semiannual updates from the vendors; plus a need to keep legacy versions alive.)

I'm tending toward a script that is launched by the user; the request placed in a queue, then performs a throttled copy of the required data to the workstation overnight. Disk space checks can be done at the point of interaction and if there's sufficient room, a dummy file created to reserve the required space. Once the file copy is complete, the user is emailed.

(The user would be "forcibly encouraged" to avoid trying to do this manually during the day by making the source file format some squirrelly thing like a cpio archive of 7z compressed chunks.)

I'm giving others chance to answer, and looking into other suggestions, before I "accept" my crappy workaround.

Rich
  • 135
0

NetApp deduplication

According to NetApp documentation, deduplication is a silent way to accomplish exactly what I need. There would simply be one "library" per user, and each user would get access to their personal "copy". ONTAP would then look after the bookkeeping.

A master copy would be kept out of sight; and this would be used to provision a new copy for each user, or freshen a user's copy.

Rich
  • 135