Background to PST files
PST files are widely used in many large organizations and they often contain business-critical or sensitive data, but they are frequently completely unmanaged. Although most administrators know roughly where to look for them, these files can be located almost anywhere and as a result, very few people actually know how many PST files they have across their organization. Often these files are located on end-user systems such as PCs or laptops, or on user drives on network servers, although they are sometimes found on removable devices such as USB drives.
In our experience, organizations typically have between two and four PST files per active mailbox. As each might possibly contain up to 10,000 messages and attachments, and about 1GB of data, there can be a very significant volume of data stored within them. If they are under end-user control, they are effectively unmanaged and probably not backed up.
Decide your Management Strategy
Firstly, you should decide your overall strategy for managing the PST files in your organization. You could choose to retain them and implement ‘in-place’ management for them, or you could migrate their data elsewhere and eliminate PST files entirely, so here is a summary of the main options.
Migrate them into Exchange or Office 365Microsoft recommends that all PST files are eliminated, as they are not a robust format and were never intended for long-term data storage. Data within PST files can be migrated back into your Exchange environment (either on-premises or in Office 365) where it is merged in with other existing email data and will be managed using existing email processes and procedures.
There are usually good reasons why users created these PST files in the first place, and Microsoft has made enhancements to Exchange and Office 365 recently to address these concerns The most significant improvement is the Archive Mailbox (or ‘In-Place Archive’) feature which replaces the old ‘Personal Archive’ feature in Outlook and allows end users to store and keep email in Exchange completely separate from their primary mailbox. It retains the previous user experience, and some Office 365 plans even offer unlimited storage for this feature.
Move them to a central network locationAnother approach is to retain your PST files, but move them from end-user locations over to a network location where they can be centrally managed and backed up.
This can be attractive as an interim solution, but it is not recommended as long-term approach because issues with network connectivity can easily corrupt PST files. The inherent design of PST files means they can generate significant ongoing network traffic and are likely give poor performance to end users, and it is important to note that Microsoft specifically recommend against taking this approach.
Migrate them into a central archiveAlternatively you could eliminate your PST files by migrating the data within them into a central third party archiving solution such as Barracuda’s Message Archiver or Cloud Archiving Service. This can be effective if you already have such a solution already in place, or if you are considering implementing one to manage your existing Exchange environment.
Keep them where they areOf course, it is possible to leave the PST files where they are and implement an application to manage them ‘in place’. This sounds attractive because there is no change for end users – they can continue to use them just as they did before. However, it’s not that simple of course because all these files still have to be identified and you must ensure that they are backed up, and they will still have the potential to become corrupted and cause problems.
Just delete themFinally, you could simply delete all the PST files your users have. Whilst this is probably not acceptable in all cases, we find that in many organizations a large percentage of the data stored by users in PST files is obsolete or no longer wanted. If you can identify which files need to be retained and which data within these files is still relevant, you can then delete the rest and greatly reduce the amount of data that does have to be managed.
Identify your Solution
Having decided your management strategy, you then need to decide what functionality you are going to implement for this, and which particular product is best to deliver this for your organization.
There are many different versions of Exchange, Windows and Outlook in common use across organizations, so make sure the product you choose supports your particular environment. Some products only support current platforms, whereas others will support older platforms back as far as Exchange 2003 and Outlook 2003.
A project to implement PST management can have up to three distinct stages depending on the strategy you choose, so it is convenient to look in turn at what’s going to be important within your solution for each of these three stages.
Stage 1 - Locate your PST files
To start with you will need to identify all the PST files in your organization. This is an essential first step regardless of how you are planning to manage them.
Most IT departments know where to look for them, as they are typically located on end-user devices and network drives - although they can of course be almost anywhere. The bigger problem is how to go about looking for them and deciding what to do with them.
ScanningIn order to discover every PST file in your organization you will need to do a thorough scan of each end user device as well as every network location where PST files are likely to be stored. Some products can also scan user Outlook profiles to locate any PST files that are currently open in Outlook, and using both approaches in parallel will increase the certainty of discovering all PST files. It can be useful to be able to restrict the scan to specified locations or paths, particularly if you already have a clear idea where PST files are located.
You will want to minimize the inconvenience to your end users. Some products require that you install a client component on each end user machine to be scanned, whilst others run purely in-memory and leave no permanent footprint. A few products take a different approach and scan end user devices remotely from a dedicated server.
OwnershipAs each PST file is located you will need to identify an owner for it and then decide what to do with it. Sometimes identifying the owner is easy (for instance, it may be attached to a particular Outlook user profile), but in other cases – such as when it is on a network drive – you may need to analyze the contents of the PST file to determine the likely owner. The more advanced products will automate this process for you.
Some organizations will want to give each end user the option to decide how their own files are managed, so the ability to provide end-user self-service is something you might look for in a product.
Stage 2 - Migrate your PST files
If you decide to migrate your PST files or the data within them, this could be into Office 365 or on-premises Exchange, into the Live mailbox or the Archive mailbox, into a third party archive, or to a specified network location. It may sound obvious, but make sure that your chosen product can actually support your chosen strategy as some provide a very limited range of options.
Minimize the impact on resourcesThere can be a very significant amount of data stored in PST files, so you will want to lessen the impact of moving this quantity of data across your network. One obvious feature to look for is a product that compresses data during transmission, and this may also include data encryption for added security. You can also reduce network traffic by choosing a product that can identify and eliminate duplicate content at source, so you are only sending a single copy of your data across the network.
Most products operate by routing or staging all data via a central server. This two-stage process can cause bottlenecks and give poor performance, so a few products take a different approach and route data directly from source to target in a ‘single hop’ avoiding their central server.
If you are migrating data back into Office 365 and Exchange th en yo u wi ll also wa nt to ma ke su re th at performance is not restricted by their throttling policies. This is a particular problem for those products which use a small number of dedicated admin logons to upload data to mailboxes. Those products that move data directly from source to target mailbox will instead use the end user logon associated with each target mailbox; this parallel processing approach is much less susceptible to throttling as well as being highly scalable, and can typically give a much better overall level of performance and throughput.
Network connections can be unreliable, and end users can shut down their machine or disconnect from the network at any time, so ensure the product is fault tolerant and robust enough to cope with these situations. It should be able to resume from where it stopped without having to reprocess any data.
Minimize the impact on end usersEnd users will often structure the way they store data in PST files, either by setting up folders within each file or by setting up a number of discrete files with structured naming conventions (e.g. by date or by project). If this is important then ensure your chosen product is able to maintain or replicate this original structure when data is moved.
It is likely that some PST files will be open within an end user’s Outlook profile, so another way to minimize the impact on end users is to choose a solution that can process PST files even if they are open in Outlook. Some products are unable to process these files, and require that the end user closes them first.
Minimize the scope of the projectThe easiest way to reduce the scope of your project is to limit the amount of data you have to move. If you don’t already have one, agree an email retention policy defining how long email data needs to be retained for, then choose a product that can look inside each PST file and delete obsolete or unwanted data based on this policy. Then you will only have to move data that needs to be kept. There may of course be some PST files that can be deleted completely, so look for a product that also includes this functionality.
You will probably need to provision additional storage for data that has been moved. If you have decided to retain PST files in a central location then you can reduce your additional storage requirements by choosing a product that will compact PST files after moving them. If you are moving data into an on-premises Exchange server then some products are able to respect mailbox quotas and will not attempt to migrate a PST file if there is insufficient room in the target mailbox.
Stage 3 – Ongoing Management
You may have chosen to retain PST files – either in their original location or in a new central location. In this case, wherever they end up your third stage will be to implement procedures to manage them on an ongoing basis.
Keep them secureThe most important consideration is to ensure that they are backed up regularly in case they become lost or corrupted. For network locations this is probably already taking place, but if files are retained on end user devices then you will need to have a robust process in place to ensure you have a secure copy stored elsewhere.
Maintain complianceOne of the key reasons for managing PST files is to satisfy information management policies. Every organization in the UK must comply with a number of different acts, and whilst none of these specify the required method of preservation, all clearly state that business correspondence, including email, must be retained and kept accessible for specific periods of time. In addition, individual organizations may put in place a ‘defensible deletion’ policy to delete some or all of their email after a specified period.
Look for a solution that enables you to implement information management policies right down to the level that you need. This can vary significantly across an organization as different departments may have different requirements, and to meet specific cases you may need to define policies based on keywords and data content as well as more clearly defined high-level metadata such as sender and delivery date. Not all solutions support this level of granularity.
Having defined your information management policies you will then need to implement them, and ideally the enforcement of these should be completely automated and applied on a scheduled or regular basis. Due to the simplistic design of PST files, it is not possible to prevent individual emails within them from being deleted prior to their retention date, so in order to ensure emails are retained your solution may have to keep a separate copy of all the data from within each PST file in a secure archive. In this case both the original email and the archive copy must be deleted automatically when the expiry date for the email has been reached.
An audit trail will enable you to demonstrate that you are fully compliant with your information management policies.
Support eDiscoveryThe other key reason for managing PST files is to support Search and eDiscovery requests. Many organizations are facing an increasing number of requests to search all existing email retained with their organization and to provide a copy of all data that meets specified criteria as well as possibly put it on legal hold.
These requests will usually want to cover live email within your Exchange server as well as data in PST files, so your chosen solution should provide an equivalent level of functionality for data stored both in PST files and in Exchange. It should identify all PST files it is currently managing so that all relevant data can be included in searches, and it should ideally be able to collect and present data from both Exchange and PST files in a single response. As with compliance, look here for a good level of search criteria and granularity, as well as the ability to refine and re-run searches.
Some requests will entail putting data on ‘Legal Hold’ as part of eDiscovery. If this is a requirement for your organization then ensure that your chosen solution includes this functionality, which will entail keeping a secure immutable copy of specified data for a specified duration.
Manage your Project
PST management can be a complex topic, so ensure you have the features to allow you to manage your whole end to end project effectively. If you are eliminating PST files altogether then it will be a one-off project and you will need progress information to manage the project through to completion, but if you have decided to retain PST files and manage them in place then you will also need to put in place ongoing performance and status reports to verify everything is operating correctly. In either situation it is also useful to have a detailed audit trail to refer to.
Some products include sophisticated processing rules that allow most aspects of your PST management operation to be fully automated. Once these more advanced products are up and running, most of your involvement will be in resolving anomalies and error situations such as corrupted PST files, as they are flagged up to you.
Whichever product you choose, we strongly recommend that you install an evaluation copy of the software and run a small proof-of-concept exercise before you go ahead and make a commitment to a particular vendor. Not only will this confirm that you have made the right choice, it will also give you valuable information to help you plan your PST management project.
About Barracuda Networks
Barracuda provides cloud-connected security and storage solutions that simplify IT. These powerful, easy-to-use, and affordable solutions are trusted by more than 150,000 organizations worldwide and are delivered in appliance, virtual appliance, cloud, and hybrid deployments. Barracuda’s customer-centric business model focuses on delivering highvalue, subscription-based IT solutions that provide end-to-end network and data security. For additional information, please visit barracuda.com.
Barracuda Networks and the Barracuda Networks logo are registered trademarks of Barracuda Networks, Inc. in the United States. All other names are the property of their respective owners.