We're hearing a series of rumors that something is going badly wrong at Dell/MessageOne. Eg:
- They've lost a huge amount of customer archived email over the past couple of weeks
- Many customers are making inquiries about other vendors and their ability to ingest/absorb their historic archive data
- One vendor told us they had been asked to help customers move their emails back from Dell/MessageOne and the most efficient way to ingest large amounts of data (10 TB for example)
Can anyone helps further? If you have input, please send to me at david.ferris@ferris.com or call me on +1 415 367 3436. If you prefer to be anonymous, let me know and we won't publish anything that identifies you.
... David Ferris
One Comment
Received this from a well-informed person: “This is symptomatic of the entire archiving market. Will be interesting to watch how bloodied Microsoft gets as a result–on-prem archiving is generally having challenges”
–David
9.30am Pacific: Dell has just contacted us, to say that it will formally respond to this post soon–David
Very long but valuable posting in from Mike Patterson, of Binary Tree, which specializes in email migration tools.
Summary of his point:”It is important that companies looking to migrate from one archiving solution to another understand the key technical factors that must be considered when doing so. By understanding the impact to the messaging infrastructure, companies can intelligently plan their migration to a new solution, whether it is a new archiving solution, reintegration of archive data into the existing platform for long-term storage, or moving to a totally new platform to take advantage of the opportunity this type of change involves.”
And now, Mike’s detailed comments:
I am a senior architect at Binary Tree. I specialize in large data migrations between platforms. I have had a lot of experience in moving large amounts of data between different messaging systems. Moving archived data from a message storage facility to the active mail system is very similar in scale to a platform migration. The most important ingredients to this equation are time, disk space on current active messaging system, reintegration utilities provided by archiving vendor, and throughput between the target and destination platforms. Let’s consider these variables.
The most important of these factors is the ability of the messaging system to receive the data, store it, and provide access to it effectively. If the messaging system is already at 90% capacity, the customer has some hard choices to make. Messaging systems such as Exchange and Domino will scale very well but for a bulk reintegration of such large amounts of data. The issue then becomes what resources are available on the target messaging servers. disk space is the overriding factor, but I/O and current system resource availability are also important. If transaction logging is in place, then the size of the partition that hold the transaction logs for either platform will be the limiting factor for moving data back into that system. In most cases, increasing the frequency of the scheduled backup tasks that manage the log volumes can take care of this limitation, but this shifts the limitation onto the backup system rather than the data storage on the email server. The second disk space factor is the size of the partition for the actual messaging data stores. If the total size of the data to be reintegrated into the messaging system exceeds the capacity of the messaging server to absorb it, then the reintegration cannot happen. If this is the case, then another archiving solution will need to be put in place to remove the reintegrated data on an ongoing basis. Like the transaction logs partition, the limitation becomes how much data can be absorbed by the second archiving system in a single period of time. The maximum size of data being absorbed will be limited to the maximum size of the transaction log partition and the frequency at which it is cleared.
Throughput between the original archive solution, the current messaging system, and the target archive solution will also limit how quickly data can be processed. In many cases, the fist day after a messaging migration is the most taxing for a network. The movement of vast amounts of data through increasingly sparse bandwidth can be very detrimental to the operations of a company. It is important to take into consideration not just the amount of data to be move, but also the straw in which the data is being sucked through between the various destinations. In most cases, daily archiving can be performed with a relatively small amount of bandwidth, but drawing the complete archived data set back into a messaging system in bulk will result in the equivalent of network asthma. Make sure that the amount of data being reintegrated into the messaging system at any one time does not exceed the capacity of the networks between the source and destination servers.
Disk I/O is also very important. The ability of the messaging system to write data to disk can also be a limiting factor, albeit small compared to disk space and network bandwidth constraints. This will also increase the amount of time necessary to reintegrate data into a messaging system. While the actual move might not tax the disk I/O of the messaging system, the access to the system for end users will be affected considerably if disk I/O capacity is exceeded the following day. In this case, the server resources will appear to go off-line intermittently while large amounts of data are absorbed and regurgitated to the end users, especially if end users synchronize data locally.
Finally, what are the tools provided by the vendor for exporting data from the archive solution back into the messaging system? Does the vendor have an administrative means of telling the system to reintegrate the messages archived back into the original message store on the active server? Is there a way of telling the archive system to reintegrate a bulk number of accounts simultaneously? Is there a way to redirect the archive data to a secondary storage facility (such as PST files or Domino archive databases)? If so, can the archive data be distributed across multiple such files in order to limit large file corruption issues? What are the requirements for such utilities to run optimized? Unfortunately, all I have for this is questions, not solutions. The only good part of this is that most vendors do have the ability to perform this action at an individual account level. The bad part is that most do not have a bulk reintegration utility for large numbers of accounts.
So what this all means, barring disk space limitations for the actual message storage partitions, is that the amount of time involved depends on what the current infrastructure can handle. Network bandwidth availability, server and archive solution system resources, and transaction log partition size will all increase the amount of time necessary to move the data from the archive solution. In addition, the availability and performance limitations of archive recovery tools provided by the vendor will also impact the time necessary to reintegrate data into the messaging store and subsequently to another archiving solution.
So what happens if the data cannot be absorbed by the current messaging system. In many cases, the archive volume far exceeds the total available disk space in a messaging environment. There are a few solutions to this issue that can be explored. The first is to evaluate the messaging system and determine a route to expand capacity in hardware for the target environment to absorb the data being reintegrated. This has a major financial risk, in that once a new archiving solution is decided upon, the disk space needed to store the vast amount of reintegrated data is no longer required. If no alternative archiving solution is sought, then this is the only way to proceed. A second solution would be to deploy the new archiving solution in the environment, reintegrate of data into the message store in phases, and allow the new archiving solution to aggressively archive the reintegrated data. This is the most fiscally sound method, but requires greater planning and administration time to complete successfully. This is the most effective means to of achieving the required results. Another possible vector would be to migrate the present data to a new solution in the cloud – such as MSO BPOS-D, Lotus Live, or Google – or an alternate messaging and archive solution. These solutions have the capacity to scale to the size necessary to bring all target data into the messaging environment. Most of these solutions can also provide an in-place enterprise archiving solution that can replace the current vendor without much additional effort. Again, these solutions still require that archived data be reintegrated into the current messaging solution, but a plan to phase in data then immediately migrate it to the new platform can also be accomplished.
We received this official response from Dell/MessageOne at 3pm PT on December 18:
“Dell is committed to delivering ongoing customer satisfaction – we are aware of the issue and are in contact with the customer who has expressed concern over Dell’s service. There are many factors associated with successful email archiving including email formatting, storage management and retention policy frequency and we are working with the customer to isolate the root cause of this issue.â€Â
This implies that just one customer has expressed concern over Dell’s service. Yet the rumors indicate that the scope of the problem is substantially beyond that of a single unhappy customer. We have asked Dell to confirm that there is nothing more to these rumors than the complaints of a single customer.
–David Ferris.
A well-informed source that wants to be anonymous for good reason says that this isn’t an isolated event, more than one customer at Dell is affected.
–David Ferris
We just completed an installation where an archive (not saying it was MessageOne) couldn’t ingest data from a 7.5TB Exchange repository – it crashed over and over again. The archive vendors have spent significant effort in building world class applications however they don’t anticipate the volume of data that needs to move into and out of the archive. As Mike Patterson mentions above – “Throughput between the original archive solution, the current messaging system, and the target archive solution will also limit how quickly data can be processed.” We have found that layering our solution, which can handle 1TB/hour processing, on top of an archive works well. Your not going to pump all the data from the email server into the archive – so use a enterprise class discovery solution to determine what to archive – and then leave the rest behind. This allows streamlined input to the archive which is more realistic versus pumping in everything and letting the archive dedupe and cull out irrelevant content.
Jim