Was Garbage Collection the Real Reason that FalconStor Selected the Violin 1010?
The last few months I have spent a lot of time looking at solid state drives (SSDs) for two reasons. One, user interest in this technology is high; and, two, any time a new technology comes out, it has its own set of issues that those selling the technology either they do not themselves yet understand or are reticent to discuss. So when the topic of garbage collection came up in the context of selecting one SSD product over another, I had to stop the conversation as I had no idea what this individual was talking about or why it mattered.
Most of the time when the topic of garbage collection comes up, I think about it in the context of two guys in blue jeans driving a big garbage truck down my street and picking up my trash every Tuesday. I certainly never think about it in the context of SSDs.
This all changed after a recent conversation I had with Matt Barletta, Violin Memory's VP of Product Management. The call came about as a result of a blog I published earlier this week on the newly announced alliance between FalconStor and Violin Memory. In that blog, I examined some of the new benefits that the Violin 1010 will provide for the FalconStor Network Storage Server (NSS) and how it will accelerate the NSS's performance.
While I see that announcement as an important milestore in the continuing evolution of the FalconStor NSS, I was a little surprised by FalconStor's decision to select the Violin Memory product. Violin Memory is not as well known as a number of other SSD providers and since the FalconStor NSS is primarily intended for enterprise environments, FalconStor had to have some pretty good reasons for selecting the Violin 1010 - hence my call with Violin's Matt Barletta.
To provide some context around why FalconStor selected Violin, Matt gave me some insight into the history of SSDs in general as he has been working in the SSD industry for some time.
Currently SSDs are being configured and resold so they look and act like hard disk drives (HDDs). Configured this way has helped to facilitate and accelerate their adoption in everything from PCs and servers to enterprise storage systems.
The problem that emerges in enterprise storage systems is that SSDs are so fast the controllers in the storage systems become bottlenecks since the SSDs process data faster than the controllers. This bottleneck has led to SSD in some enterprise environments moving out of the storage arrays and back into the servers.
However another lesser known problem about SSDs that starts to show up as they start to fill up their available capacity limits is their inability to effectively do erases. An SSD cannot erase just a single location or page but has to erase an entire block that consists of 2000 pages. Therefore to erase a single page, it has to copy all 2000 pages in the block to another location on SSD, erase the data on the source block and then rewrite the 1999 pages that are to remain back to the original location. This entire copy, move and copy back process is known as garbage collection.
The good news is that this garbage collection only takes about 3 milliseconds to complete which is approximately the same amount of time than it takes for a hard drive to spin up. So as long as this garbage collection does not occur too frequently, the performance impact on the application is minimal.
The bad news is that due to the current high cost of SSDs, they are starting to be deployed as a cache in front of network storage arrays (aka the FalconStor NSS).This is done as a means for organizations to optimize their use of SSDs for their most active data.
Used in this manner likely means that inactive data will have to be frequently deleted from the cache to make room for more active data. But unless this garbage collection is handled efficiently, substantial drop-offs in application performance can occur as the SSD starts to thrash doing this garbage collection for tens or hundreds of blocks.
This sets the stage to explain why FalconStor may have selected the Violin 1010 memory system over competing SSD solutions. Violin was developed, in part, on the premise that SSD would initially be used in enterprise environments not as a primary data store but as a cache. Used in this way also meant that SSDs would start to experience this thrashing phenomenon that results when garbage collection is occurring.
In the background, Violin manages this garbage collection so there is no detectable drop-off in application performance while the garbage collection occurs. So while other reasons are cited in FalconStor's press release for selecting Violin (the Violin 1010's performance, availability characteristics and ability to support hot swaps of failed DRAM components), the hidden reason for FalconStor's selection of Violin may ultimately be Violin's ability to do garbage collection better than anyone else.
While this conversation was the main item that piqued my interest this past week, one other piece of news also caught my attention that came in the form of email from my friend Ken Schneider, the President at Perm-A-Store. The news itself was nothing earth-shattering but companies like Perm-A-Store provides a good reality check for me, especially in the area of portable media such as disk and tape.
In this particular instance, I was reading between the lines in an email that he wrote to the Bare Metal Data Group and, from it, I can begin to detect that tape's demise is getting closer but the future for offsite records management firms is not as bleak as one might think. Ken writes:
The future of tape is an awfully big question for this group (and especially me!!!). We (Perm-A-Store) are seeing slight decline in tape case sales and quite a bit of activity in cases for removable hard drives and I wonder about Flash (SSD) as another replacement.
The upside in his email is that even as we are starting to see the end of one portable technology others such as removable hard disk drives and flash seem poised to take tape's place.
Have a good weekend everyone!
Most of the time when the topic of garbage collection comes up, I think about it in the context of two guys in blue jeans driving a big garbage truck down my street and picking up my trash every Tuesday. I certainly never think about it in the context of SSDs.
This all changed after a recent conversation I had with Matt Barletta, Violin Memory's VP of Product Management. The call came about as a result of a blog I published earlier this week on the newly announced alliance between FalconStor and Violin Memory. In that blog, I examined some of the new benefits that the Violin 1010 will provide for the FalconStor Network Storage Server (NSS) and how it will accelerate the NSS's performance.
While I see that announcement as an important milestore in the continuing evolution of the FalconStor NSS, I was a little surprised by FalconStor's decision to select the Violin Memory product. Violin Memory is not as well known as a number of other SSD providers and since the FalconStor NSS is primarily intended for enterprise environments, FalconStor had to have some pretty good reasons for selecting the Violin 1010 - hence my call with Violin's Matt Barletta.
To provide some context around why FalconStor selected Violin, Matt gave me some insight into the history of SSDs in general as he has been working in the SSD industry for some time.
Currently SSDs are being configured and resold so they look and act like hard disk drives (HDDs). Configured this way has helped to facilitate and accelerate their adoption in everything from PCs and servers to enterprise storage systems.
The problem that emerges in enterprise storage systems is that SSDs are so fast the controllers in the storage systems become bottlenecks since the SSDs process data faster than the controllers. This bottleneck has led to SSD in some enterprise environments moving out of the storage arrays and back into the servers.
However another lesser known problem about SSDs that starts to show up as they start to fill up their available capacity limits is their inability to effectively do erases. An SSD cannot erase just a single location or page but has to erase an entire block that consists of 2000 pages. Therefore to erase a single page, it has to copy all 2000 pages in the block to another location on SSD, erase the data on the source block and then rewrite the 1999 pages that are to remain back to the original location. This entire copy, move and copy back process is known as garbage collection.
The good news is that this garbage collection only takes about 3 milliseconds to complete which is approximately the same amount of time than it takes for a hard drive to spin up. So as long as this garbage collection does not occur too frequently, the performance impact on the application is minimal.
The bad news is that due to the current high cost of SSDs, they are starting to be deployed as a cache in front of network storage arrays (aka the FalconStor NSS).This is done as a means for organizations to optimize their use of SSDs for their most active data.
Used in this manner likely means that inactive data will have to be frequently deleted from the cache to make room for more active data. But unless this garbage collection is handled efficiently, substantial drop-offs in application performance can occur as the SSD starts to thrash doing this garbage collection for tens or hundreds of blocks.
This sets the stage to explain why FalconStor may have selected the Violin 1010 memory system over competing SSD solutions. Violin was developed, in part, on the premise that SSD would initially be used in enterprise environments not as a primary data store but as a cache. Used in this way also meant that SSDs would start to experience this thrashing phenomenon that results when garbage collection is occurring.
In the background, Violin manages this garbage collection so there is no detectable drop-off in application performance while the garbage collection occurs. So while other reasons are cited in FalconStor's press release for selecting Violin (the Violin 1010's performance, availability characteristics and ability to support hot swaps of failed DRAM components), the hidden reason for FalconStor's selection of Violin may ultimately be Violin's ability to do garbage collection better than anyone else.
While this conversation was the main item that piqued my interest this past week, one other piece of news also caught my attention that came in the form of email from my friend Ken Schneider, the President at Perm-A-Store. The news itself was nothing earth-shattering but companies like Perm-A-Store provides a good reality check for me, especially in the area of portable media such as disk and tape.
In this particular instance, I was reading between the lines in an email that he wrote to the Bare Metal Data Group and, from it, I can begin to detect that tape's demise is getting closer but the future for offsite records management firms is not as bleak as one might think. Ken writes:
The future of tape is an awfully big question for this group (and especially me!!!). We (Perm-A-Store) are seeing slight decline in tape case sales and quite a bit of activity in cases for removable hard drives and I wonder about Flash (SSD) as another replacement.
The upside in his email is that even as we are starting to see the end of one portable technology others such as removable hard disk drives and flash seem poised to take tape's place.
Have a good weekend everyone!
Buyers Guides
Newsletter Signup
Interesting question about removable Flash devices replacing tape for long term archiving. I have never heard or seen any analysis of whether Flash devices have any issues storing static data for long periods - years or decades.
Anecdotal evidence from using consumer-grade flash thumb drives and memory sticks, etc, indicates that the passage of several/many years has no effect on readability of the media... my oldest thumb drive (a once gigantic 32MB unit) is still readable, though I haven't written anything to it for many years.
In a related but semi-off-topic counterpoint, my wife just encountered several unreadable DVDs (DVD-R media) she had archived some business files onto in 2006 and 2007. I was very disappointed to find this longevity issue, although it appeared to only be one DVD-R manufacturer/brand that was failing, other types could still be read, thankfully.
But it was a wake-up call to me that, even though something writes and verifies successfully, and can be read again several months or a year or two later, you must occasionally test older archives for readability - a step that is frankly missing from many archive schemes.