Data Verification

The BSI system will enforce referential integrity and other constraints on the data as it is entered and modified. BSI verification checks in conjunction with good laboratory practices can guarantee that your data is clean when you want to query it.

Edit Checks to Keep Your Data Clean

Data verification or edit checks are the cornerstone to valid data in a database. There is extensive diversity in the edit checks that BSI supports. They can range from simple syntactical checks (e.g. the value must come from a code list) to complex checks that combine data from several sources (e.g. the amount of serum collected during the third visit must be greater than 5.0 ml). A typical data entry record will undergo 200 different verification checks before it is allowed to be added to the BSI database. In addition, BSI also allows you to double key data into the system. This allows you to eliminate many keying errors prior to committing that data to the database.

Templates to Provide Automated Data Entry

BSI provides data templates to automatically generate data when entering or modifying records in the database. Templates can define default data for a single specimen record or a series of records. BSI also provides special formulas that templates can use when generating new records. Templates and formulas can be used during direct data entry or when you are importing data from a file.

Consistent Data Formats with Normalization

BSI can also normalize your data for you. Normalization will convert your entry into a format specified by you. Examples include capitalization, stripping leading and trailing white space, and ensuring that numeric portions of alpha-numeric fields are converted to a consistent format. Data normalization is also utilized with categorical fields, allowing you to enter the code or the label for a value. Normalization is implemented throughout the BSI system, but is most pertinent during data entry and searching.

The ROI for Clean Data

The cost of clean data is difficult to quantify, but many studies have shown that going back to clean data after collection is expensive. For example, IMS recently had to import a data set of over one million specimens into BSI for a new customer. This customer had previously stored this data in a dBase-style database that contained only inventory information. This database was similar to a set of Access tables with no verification checks. The data was corrupt on several levels. There were many problems, such as duplicate primary keys, garbage fields, duplicate records, missing records, etc. It took three months and $45,000 (IMS costs) to clean the data sufficiently to import it into BSI. There were also more costs on the client side to research and resolve discrepancies. Assuming that the client side costs were the same, going with an Access type system just cost $90,000 to clean the data.

Keeping your data clean is imperative and will save you money in the long run.