vCenter Crashing - SQL Express DB reached licensed limit of 10GB

Twice in a week vCenter crashed during Veeam backups.  Exploring the application event log of the vCenter server revealed the MS SQL errors.

"Could not allocate space for object 'dbo.VPX_EVENT_ARG'.'PK_VPX_EVENT_ARG' in database 'vCenter' because the 'PRIMARY' filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup."

"CREATE DATABASE or ALTER DATABASE failed because the resulting cumulative database size would exceed your licensed limit of 10240 MB per database."

SQL Express is limited to 10GB per database and over time the event history will fill this space.

By default the event history is kept indefinitely, but this can be changed.

Exerpt from the VMware KB Article Purging old data from the database used by VMware vCenter Server 4.x and 5.x (1025914)

To configure these Database Retention Policy settings to prevent reaching the limit, use the setting in the vSphere Client, click Administration > vCenter Server Settings > Database Retention Policy. Set the days of retention appropriately.

Follow these steps for Manual Resolution:

To purge the data in the VPX_EVENT table:

• Connect to Servername\SQL Database and log in with the appropriate credentials.
• Click databases to expand and select VIM_VCDB > Tables.
• Right-click the dbo.VPX_PARAMETER table and select Open. (If you are using SQL Server 2008, right-click the dbo.VPX_PARAMETER table and click Edit Top 200 Rows.)
• Modify event.maxAge to 30, and modify the event.maxAgeEnabled value to true.
• Modify task.maxAge to 30, and modify the task.maxAgeEnabled value to true. 

Note:To improve the time of the data cleanup, run the preceding steps in several intervals. To do this, ensure to keep the default value of event.maxAgeand task.maxAgeand perform step 6 to run the cleanup. Then, reduce the event.maxAgeand task.maxAgevalue by 60 and run the cleanup. Repeat the steps until the value is reached to 30 for the final cleanup process.

Run the built-in stored procedure:

• Go to VIM_VCDB > Programmability > Stored Procedures.
• Right-click dbo.cleanup_events_tasks_proc and select Execute Stored Procedure.

This purges the data from the vpx_event, vpx_event_arg, and vpx_task tables based on the date specified for maxAge.

When this has successfully completed, close SQL Management Studio and start the VMware Virtual Center Server service.

To ensure that the default Statistics Level is set to 1:

• Using the vSphere Client, log into vCenter Server as an administrator.
• Go to Administration > vCenter Server Settings > Statistics.
• Under Statistics Intervals, ensure the Statistics Level column is set to 1.
• To change the value, select the Interval Duration, click Edit and select Level 1 from the dropdown.