Missing Database Records

Posted: Fri Jun 15, 2007 1:49 pm Post subject: Missing Database Records

Hi all
I have a database which I load every month as a CSV using PHPMyAdmin. The datbase has 10,000 records and I loose anywhere between 10-40 records whilst loading. Is there a way, to find which records have been rejected? I use an auto-number to generate my key index, so it's not a duplicate key problem.
Any thoughts?

NetHosted Staff Joined: 22 Mar 2004 Posts: 5625

Kev,

There isn't anything server side I can think of to find this out.

Andrew

Posted: Fri Jun 15, 2007 6:47 pm Post subject:

Could you re-export to CSV and use a diff tool (ie: Beyond Compare) to identify differences?

Also, since this is a large monthly task, you might consider writing a specialist tool for it, that reads in the CSV and logs any errors as they happen?

Posted: Sat Jun 16, 2007 10:02 am Post subject:

This reminds me of an issue I had reading raw records from webpage logs. Mostly the records read in fine, but occasionally someone would try to log in with a space in their login name. This would be faithfully recorded in the log but it would mess up my separation character settings and cause no end of problems. I found this by reading the csv into a MS Access database first, which politely isolated the records it could not make sense of, I was able to fix it up and then send it along to mysql.