Exchange DAG Node down, Windows Cluster Service will not start

Problem: After a server reboot an Exchange DAG member is down, the Cluster Service Fails to start.

Unable to start the cluster service it terminates with the Event ID 7024

"The Cluster Service terminated with service-specific error cannot create a file when that file already exists." 

Error Messages in Event Log:

Service Control Manager Event ID 7001

The Cluster Service service depends on the Csv File System Driver service which failed to start because of the following error: 
The system cannot find the file specified.


Service Control Manager Event ID 7000

The Csv File System Driver service failed to start due to the following error: 
The system cannot find the file specified.



MSExchangeRepl Event ID 4113

Database redundancy health check failed.
Database copy: DB_NAME
Redundancy count: 1

Error: Passive copy 'DB_NAME\EXCH_SERVER' is not UP according to clustering.


MSExchangeRepl Event ID 2060

The Microsoft Exchange Replication service encountered a transient error while attempting to start a replication instance for DB_NAME\EXCH_SERVER. The copy will be set to failed. Error: The NetworkManager has not yet been initialized. Check the event logs to determine the cause.

Solution:

Check Device Manager and the Microsoft Failover Cluster Virtual Adapter has a yellow exclamation mark (this is netft.sys)

“This device is not working properly because Windows cannot load the drivers required for this device. (Code 31)”

Need to remove the "Microsoft Failover Cluster Virtual Adapter" and reinstall it using the following steps: 

1. From Device Manager/Network adapters, click on View-->Show hidden devices and then uninstall "Microsoft Failover Cluster Virtual Adapter" 
2. Reboot the server for changes to take effect 
3. After reboot Open Device Manager/Network adapters 
4. From Action Menu Select “Add Legacy Hardware” and then click Next 
5. Select “Install the Hardware that I manually selected from a list (Advanced)” and click Next 
6. Select “Network Adapters” and then click Next 
7. Select “Microsoft” From the left pane and select “Microsoft Failover Cluster Virtual Adapter” from the right list 

8. Once the Adapter is added successfully you be able to start the cluster service successfully and let the DAG sync up the queue log files.

NOTE: If you reboot your server and issue comes back follow the same steps but disable IPv6 on the network adapters since it is not required when Exchange is not on a Domain Controller.

VEEAM - Tips for DAG Exchange Backup and Replication in vSphere

If you have tried everything to keep your Exchange DAG in balance based on datastore AP (Activation Preference) here is a quick PowerShell that will help you.

I have tried the Veeam KB http://www.veeam.com/kb1744 no luck still 1 out of 8 for that site failed over.

Tweaked DAG Settings as recommended (this helped by keeping 1 or 2 databases others failed over)

1. cluster /prop SameSubnetDelay=2000:DWORD     (Default: 1000)
2. cluster /prop CrossSubnetDelay=4000:DWORD     (Default: 1000)
3. cluster /prop CrossSubnetThreshold=10:DWORD     (Default: 5)
4. cluster /prop SameSubnetThreshold=10:DWORD     (Default: 5)

From PowerShell as Admin:

. 'C:\Program Files\Microsoft\Exchange Server\V15\Scripts\RedistributeActiveDatabases.ps1' 'YOUR-DAG-NAME-HERE' -BalanceDbsByActivationPreference -Confirm:$false



Or Script it in Task Scheduler

Program/script: powershell
Add arguments: -command "& 'C:\Program Files\Microsoft\Exchange Server\V15\Scripts\RedistributeActiveDatabases.ps1' 'YOUR-DAG-NAME-HERE' -BalanceDbsByActivationPreference -Confirm:$false"




NOTE: Run this a few hours after the backup, because when the DAG will fail the Content Index State will go to FailedAndSuspended then followed by Seeding state. After it has returned to Healthy then the script will run without errors. So run this before business hours start or middle of the night after the Index state state has returned to healthy. You can write another PS to query the state once all DAGs are healthy then balance them out.


Hope this will help someone if it does +1 or send a comment.

Cheers,
Jakub 

Samsung EVO 850 vs Intel SSD 520 Series

I decided to do a small upgrade on my older Lenovo T430s by increasing the SSD capacity. Based on reviews I purchased a Samsung EVO 850 256GB SSD to replace the Intel SSD 520 180GB. Small space upgrade the 180GB was not enough for all the programs, tools, images and large PST files.

NOTE: Keep in mind this an older generation Intel SSD 520. Intel SSD 730 would be on the same generation level as the Samsung EVO 850


Intel SSD 520 Series (180GB)

Samsung SSD 850 Pro (265GB)

RAPID Mode
After installing Samsung Magician and enabling RAPID (Real-time Accelerated Processing of I/O Data) mode. It did give a nice boost because uses the systems RAM to cache hot data. This is show during the 4 KB random multi WRITE test as the mutli writes can not be cached it crippled a bit. Nice free RAM drive hot cache tool from Samsung working inside windows only.






Summary

QuickBooks - An error occurred. Your QuickBooks installation may have been modified. Reinstall QuickBooks and try again. If the problem persists, contact Intuit Technical Support and provide them with the following error codes: (-6010,-100).

When you launch quickbooks, select your company profile, put in the username and password you get the following error message: 

An error occurred. Your QuickBooks installation may have been modified. Reinstall QuickBooks and try again. If the problem persists, contact Intuit Technical Support and provide them with the following error codes: (-6010,-100).

You can try to reinstall, call QB support all options will get you no where trust me! 

I tried to fix the service switches / arguments and no luck with getting the service to start with the parameters I wanted. Quick fix for now is to execute the following from your QuickBooks directory:

"QBDBMgrN.exe -n QuickBooksDB24"

ie. "C:\PROGRA~2\Intuit\QUICKB~3\QBDBMgrN.exe -n QuickBooksDB24"

The SQL Anywhere interface will start and minimize, make sure not to log off from the server. Best is to run this on the console or as a scheduled tasks with an IF statement so one executable is running.

Now you can log into quickbooks.

P.S. To Intuit, for those users complaining about slow quickbooks wonder no more. The Sysbase database engine is licensed to use 2 physical processor, what about the cores ? Thanks Intuit!

P.S.S. If you want to see other options to for the service execute the tile as "QBDBMgrN.exe -?" the service option had -hv as a parameter, no explanation in the help page.

VMware Horizon View - Error occurred during vCenter operation.

Came across an annoying message and only possible fix I found was to recreate the pool. Thinking that is a bit extreme. When I edit other VMware View 5 Horizon pools no error messages. Rebooted vCenter and the View Connection server with no resolution and no errors in the event logs.

Nothing has changed on vCenter, datastores etc.. for months. The Parent VM and snapshots were all there.

Error Message (Editing a Pool):

"Error Occurred during vCenter operation."

Error Message (Save Pool Settings):

"Could not find the parent VM or snapshot"

Solution:

Select the affected pool, click EDIT, vCenter Settings tab.

For each of the Browse... options click browse and select the same object.

Once you have gone through all the steps click OK and now you can go back and edit the pool.

P.S. If you had to rebuild vCenter Server just refresh the pool and it will all be back to normal.

NetGear ReadNAS - Bad Magic Number in Super-Block

Hardware: NetGear ReadyNAS 3200

fsck.ext4: Bad magic number in super-block while trying to open /dev/c/c
/dev/c/c:
The superblock could not be read or does not describe a correct ext2 filesystem. If the device is valid and it really contains an ext2 filesystem (and not swap or ufs or something else), then the superblock is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>

Solutions: 

Get a new ReadyNAS. Spend weeks with support engineers from Europe and USA, your data is gone... new unit required.

The Microsoft Exchange Information Store service terminated with service-specific error %%-2147221213.

Problem:

Exchange 2010 DAG has a very high copy queue length.

Non-active node can not start Microsoft Exchange Information Store Service or System Attendant Service.

System Event log error message:

The Microsoft Exchange Information Store service terminated with service-specific error %%-2147221213.
Event ID: 7024

Solution:

1. Make sure your Windows Time Service is running (optional)
2. Note the time difference between your current local clock and domain clock.
2. Run: cmd -> "net time /set" this will pull time from your local ADC site.
3. Shutdown the VM, do not power on.
4. Check the hypervisor clock to make sure it's correct and make sure that the guest VM does not sync time with the host.
5. Wait for the time to pass noted in step 2 and power on the VM.
6. Check Microsoft Exchange Information Store and System Attendant service should be running now.