Any school that has had to correct erroneous data that affected,
or could have affected, its placement in the rankings knows that
ensuring the accuracy of data submitted to outside sources is
critical. Regardless of whether the misreporting was intentional or
the product of an oversight, the act can have serious
repercussions, not the least of which is damage to a school's
public image. Fortunately, there are simple and cost-effective
actions that an institution can take to avoid releasing inaccurate
data. The following are best practices that should be considered to
avoid misreporting university data to external sources and
surveys.
Maintain a Central Depository of the Data Used to Calculate
Externally Reported Metrics by Utilizing a Standard Student Record
System
Externally reported data should be generated from an
institution's standard student record system, which undoubtedly
includes various data security checks and balances. Using the same
record system across all schools, including graduate and
professional schools, allows for centralized review and
contemporaneous tracking of the information used in data reporting.
All actions affecting data in the system, such as who loads data or
makes changes to data entries, will be tracked so that any changes
may be reviewed and audited in the course of responding to external
survey requests. Exclusive access to the system by username and
password ensures that any additions, deletions, or alterations to
data can be attributed to a specific user.
Consider a Centralized Process for Reporting Data in
Response to External Survey Requests
To the extent an institution has a decentralized process for
reporting data to external survey requests, it should consider a
centralized process or implement centralized procedures in order to
verify the accuracy and consistency of data. The mandated
involvement of a last-stop, quality-control office ensures that any
outgoing data will be reviewed a final time before it is
disseminated externally. In addition to its role as the final
gatekeeper for the thoroughness and accuracy of data, such an
office would serve as the first and last point of communication for
any outside entity soliciting data from the institution.
Questions from data source offices (i.e., the office of admission,
the office of financial aid, etc.) should be directed to the
centralized reporting office, thereby promoting consistency in the
feedback and instruction to all data source offices. Another task
for the centralized reporting office should be to conduct frequent
and random audits of the data it receives. If errors are detected,
the data should be returned to the data source office for review
and revisions, and the data review process should start anew.
Finally, the centralized office should maintain a data warehouse
where, among other data, copies of all externally reported data are
preserved.
Various Persons at the Data Source Office Should Review the
Data for Accuracy
Data should be critically reviewed by employees within a data
source office. Any assumptions made in calculating the data should
be put in writing and examined by a senior decision-maker within
the data source office before the data is transferred to a central
reporting office. The onus to provide accurate data is with the
employees of the data source office since they are the ones who are
most familiar with the data and its collection.
Despite the importance of accurate data analysis, the task is
sometimes relegated to employees with low seniority due to the fact
that data calculation can be time-consuming and tedious. Such
employees rarely have the level of statistical training or
background that would compensate for their lack of experience. In
cases where a junior employee takes the first pass at data
analysis, the data should be reviewed by additional persons in the
data source office, including at least one person of mid to high
seniority. But in all instances, data should be reviewed by more
than one person, as a practical internal control at the ground
level.
Segregate Duties to Avoid Creating Conflicts of Interest
and to Create Independent Verification and Accountability in
Reporting
Persons whose performance is measured, even in part, by the results
of any operation (e.g., the ability to recruit a certain number of
students while maintaining existing admission standards) should not
also be responsible for accounting for the results of those
operations and should not be the final and ultimate reporting
source. Place the responsibility for accounting for and reporting
the results of operations with persons who are not evaluated by,
and have no direct interest in, the results of those
operations.
The unencumbered reviewer should review the reports he or she
receives at an appropriate level of detail prior to their
transmission to a central reporting office or to third parties.
Each person involved in the review process should have a clear
understanding of his/her responsibilities. The institution should
develop a review checklist, which evidences the completion of each
required review procedure. This checklist should be completed prior
to transmission of any report to third parties. A copy of the
completed checklist should be maintained in the centralized data
warehouse with a copy of the transmitted report.
Create Clear, Unambiguous Policies and Related Processes
for All Significant Operations
A clear, unambiguous written policy will guide personnel as they
prepare responses to external surveys. Clear policies also protect
against lost institutional knowledge in the event of personnel
changes. The policies not only need to be explicit, but all
personnel involved in the preparation of responses to external
surveys must be aware of the policies and how they can be accessed,
and that the institution is dedicated to following the processes
set forth in the policies. Ambiguities in policies to be applied in
making significant reporting decisions can lead to the
rationalization of inappropriate choices. Developing and
communicating clear-cut policies about recordkeeping and survey
answering will lead to better quality decision-making and
accountability.
Document Judgment Calls Being Made Contemporaneously with
Rationale for the Decision
In the event an external request for numerical data is ambiguous
and could reasonably and ethically be answered by more than one
method of calculation, document any judgment calls relating to how
an answer was determined. Record precisely the means by which the
answer was calculated, why such an approach was taken, and who was
responsible for making such decisions.
To the extent the data provided to the central reporting office
relies on any assumptions made by the data source office, the
central reporting office should be made aware of such assumptions
and given the opportunity to accept or reject them before verifying
the accuracy and consistency of the data for external
distribution.
Consider the Implementation of a Code of Business Ethics
and Conduct Policy and Hotline to Foster and Promote a Culture of
Integrity
Assuming such a policy is not already in place, consider the
implementation of a Code of Business Ethics and Conduct policy that
informs employees of their professional duty to do their job
honestly and ethically. In conjunction with this policy, staff
should be trained annually and sign their acknowledgement and
adherence to the policy. Training should emphasize each
employee's duty not only to act in accordance with the policy,
but to report other employees' violative conduct, even if a
superior is involved. To implement this part of the policy, an
institution may choose to utilize a telephone hotline to which
anonymous calls may be made. Employees should not only be obliged
to report conduct that they know is violative of the policy, but
they should also be encouraged to report conduct over which they
have concerns. The policy and hotline information should be highly
visible, posted on the intranet, and in well-trafficked employee
areas.
If Incorrect Data Is Reported Externally, Correct the
Erroneously Reported/Disseminated Data and Conduct an Assessment of
Risks Related to Other Data
Identify and correct any erroneously reported or otherwise released
data. Where possible and after appropriate levels of review,
contact the party to whom the data was reported and provide it with
the corrected data. Once it is determined that an institution has
reported erroneous data, it is recommended that the institution
conduct a risk assessment as to the accuracy of other reported
data. A key component in the risk assessment is determining what
aspect of the internal controls fell short.
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
