Guest post by Along with Storage Spaces coming in Windows 8, ReFS forms the foundation of storage on Windows for the next decade or more. Key features of the two when used together include: metadata integrity with checksums; integrity streams providing optional user data integrity; allocate on write transactional model for robust disk updates; large volume, file and directory sizes; storage pooling and virtualization making file system creation and management easy; data striping for performance and redundancy for fault tolerance; disk scrubbing for protection against latent disk errors; resiliency to corruptions with “salvage” for maximum volume availability in all cases; and shared storage pools across machines for additional failure tolerance and load balancing.
Although some nice features of NTFS will be missing it uses some current FS “best practices” like B-trees, Copy on write, integrity checks and extends (like ZFS and BtrFS)
MS has gone for complexity reduction (reduced feature set and NTFS code sharing) because otherwise you cannot ship a new production ready file system.
That it shares a lot of code with NTFS could be nice for a reimplementation in alternative operating systems.
NTFS has all of these, but copy-on-write. SIS and VSC use something similiar, though. And unlike ZFS and BtrFS NTFS also has a working undelete function
Other than that I agree, ReFS looks good. Let’s wait and see where it is going.
BS.
NTFS is not B-tree based, has no file integrity checks and no extends and it misses a lot more features ZFS and BtrFS have.
Read the Wikipedia or something.
Edited 2012-01-20 08:15 UTC
Don’t worry, ReFS is most likely to be scrapped before the Windows 8 Server release. Don’t mod me down, you know it’s true.
Actually I know they’re going with it. Linux will have btrfs (Oracle will make sure of it), Solaris and FreeBSD have ZFS so Windows Server is left waaaaaay behind and MS does not want that.
Aint competition a great thing ?
After Vista launched with lots of features removed, Microsoft brought the Office team to work on Windows. Ever since then, whenever they announce and demo a feature, you can bet it’s going to pop up in the operating system. So, yes, it is going to be in Server.
Edited 2012-01-17 15:42 UTC
FTA:
Some of those are useful features, even if they are underused.
True. But didn’t NTFS go through several iterations? Wikipedia says five versions:
Nothing’s stopping Microsoft from doing something similar again.
That’s my thought as well. As they implement ReFS onto the client (perhaps with Windows 9, perhaps with a service pack to Windows 8) they’ll have to add important features that clients use.
For the server environment, the features they removed won’t be missed considering Storage Pools and ReFS (with copy on write semantics) supersede them. Server 8 will have deduplication, which takes care of the need for Sparse files, hard links, and compression. It’s probably needlessly complex to support all the dropped features considering the new file system and storage pool.
But for the client, Sparse files are critical because large files would take forever to allocate on typical HDDs. Hard links I’m not so sure about, because they are rarely used in Windows, but they are still nice to have for the client without dedup.
Finally, removing those other features is actually a good thing considering malware. Often all those obscure NTFS features are used to hide malware so deep in NTFS only specialized tools can find them.
Edited 2012-01-17 16:54 UTC
Citation needed. I’ve just read exactly the opposite for ReFS(at least out of the box, it seems to be avail via third party addons)
Edited 2012-01-17 17:06 UTC
http://social.msdn.microsoft.com/Forums/zh/windowsdeveloperpreviewg…
It’s implemented as a scrub job that runs in the background to deduplicate data after the fact.
Edited 2012-01-17 17:19 UTC
RTFA:
Hmm, a tradeoff I suppose.
NTFS supports deduplicatation in server 8.
I take back what I said about ReFS then. It doesn’t supercede all the needed features.
EDIT:
Thinking about it for a minute, it does seem kind of a weird omission considering the support is already there for NTFS for dedup.
But I guess the reason they decided to not support it is because it goes against having redundancy in the system. They specifically mention you only get checksum repairing when using mirrored storage pools, not parity. Though with parity errors are still logged with checksums.
Still, it leaves an interesting opening for dedup while using ReFS. I know there is software that allows you to layer one file system on top of another. Presumably, you could layer an NTFS volume on top of ReFS to get advantages of both, but probably suffer a performance hit.
Or just wait for Microsoft to implement it in future versions.
Edited 2012-01-17 17:40 UTC
Dedupe is not built into NTFS on Windows 8. It’s a separate “thing” that runs on top of NTFS.
http://blog.fosketts.net/2012/01/03/microsoft-adds-data-deduplicati…
Edited 2012-01-17 19:52 UTC
more reason to predict the enthusiasts will be using windows 8 server!
I just have to:^)
One year back..
http://www.osnews.com/permalink?456042
http://www.osnews.com/thread?456047
Excuse the assinine attitude in the second comment.
Edited 2012-01-17 18:34 UTC
Heh!
I really like the work going into bit rot protection through checksums, this is the number one feature of ZFS for me. Yes de-dup and superior RAIDZ capability are really useful as well as snapshots. However bit rot has to be my number one worry as it’s so easy to reproduce, through bad RAM, faulty controllers on the RAID Card, HDD controller etc.. so i think these checks are a minimum requirement of our information society.
Slightly disappointed not to see de-dup, but this is a version 1 and ZFS went through quite a few versions to get where it is today.
Ill have to reserve judgement on the storage spaces concept until i see how it works in the real world.
My only worry is that each new feature added to ReFS will require a new version of Windows, i really hope that Microsoft can backport the features through updates and Service Packs, as i wouldn’t want to keep going through each server to get fundamental file system features. Generally in today’s Windows word you can work on the other principle, Windows 2003 skip 2008 2008R2, for some features it might compel me / us to go for 2012.
Im also a little worried how this will work with Windows 7 / 2008R2, i personally think Microsoft could get a lot of good will by backporting this to Windows 7/2008R2.
Obviously in an ideal world, the licence would be free and allow other OS’s such as MacOSX and Linux to take advantage of it aswell, but i don’t see it happening.
It seems that MS ReFS only has checksums for metadata, so the data itself might still be corrupted. ReFS exposes an API so a vendor can use checksums on data, but that is not default behavior.
Thus, ReFS seems safer than NTFS, but will it be safe enough?
From the article:
It’s the default behavior when redundancy on spaces is present. When that is true, we can use the data checksum to find a good copy if one exists and another is bad. Without redundancy, all we can do with the checksum is prevent bad data going to applications – ie., start failing requests. Since applications will not always deal with that gracefully, the benefit without redundancy is much more limited – where failure really is better than incorrect data.
I realize many people might want to get religious on this point, but seriously, watch what happens when reads are failed under applications first. Particularly consider what happens when a read for a block of code is failed, for example. Then consider the fraction of that code that will actually be executed.
– M // ReFS co-author
If you’ve read the news lately, you’ll know that “software patents” are a main key there.
Hey, getting free money from other people, like LG, even if LG invented what they are using.
If you want to write a reader for that new filesystem, you know you’ll have to…
Edited 2012-01-19 18:17 UTC
they are dropping support for (among others) hardlinks, quota, encryption, compression, sparse files….
I don’t get it ?? rather very usefull features which all/most FS have… and MS drops them ?
companies need quota
efficient backups need hardlinks
cheapos need compression
government does not want encryption (ah..?)
sparse files: very useful for databases
nop… I don’t and won’t get it in this form :/