Key Statistical Issues In Eye Doc Fraud Sentencing

The court found flaws in both the government's and defendant’s analyses and, as a result, calculated its own loss figures in determining a reasonable loss amount for sentencing. Based on this calculation, the court found an intended loss of over $73 million. This article provides an overview of that case, along with a technical summary of the key statistical issues and the court’s ultimate findings.

Case Overview

In April 2017, Dr. Solomon Melgen was convicted by a jury on charges that he engaged in a scheme to defraud Medicare and other health care benefit programs through his clinic, Vitreo-Retinal Consultants. Melgen was found guilty of routinely diagnosing patients with wet or dry age-related macular degeneration and then submitting false claims to Medicare and billing for medically unnecessary procedures such as laser surgeries and eye injections.

As part of its initial investigation, the government randomly selected a sample of 310 patients from the total population treated by Melgen between January 2010 and December 2012. That sample was subsequently examined by government investigators and medical experts.[1] The investigation led to the 2015 indictment against Melgen and a subsequent trial and jury verdict concluding in April 2017. Upon conviction of all charges by the jury, the government sought to extrapolate its findings from the sample to estimate losses for the purposes of sentencing. Multiple sentencing hearings were conducted in December 2017 and January 2018, and the court ultimately sentenced Melgen to a term of imprisonment of 204 months on Feb. 22, 2018.[2] On March 8, 2018, Melgen filed a motion for bond pending appeal and motion for new trial in the district court.[3] That same day, Melgen filed a notice of appeal in the Eleventh Circuit.[4]

The Government’s Extrapolation

Sample Selection

Between January 2010 and December 2012, Melgen treated a total of 1,606 unique Medicare patients (the “original population”). The government sought “to have a statistically significant sample drawn, so that any findings with respect to this sample group could be extrapolated to the whole group of 1,606.”[5] Subsequently, three samples of 310 patients each, labeled A, B and C, were randomly selected. Following a preliminary analysis, the government ultimately chose sample C for further investigation.

Review of the Sample

The government’s medical experts reviewed Melgen’s medical and billing records for each patient in sample C. Experts evaluated, among other things, whether the patient records supported or refuted specific diagnoses reflected in Melgen’s billings for each respective patient. Ultimately, the government determined the proportion of unsupported billings associated with each diagnostic code. For instance, it alleged that 84 percent of the sample’s billings submitted under diagnostic code 362.52 were unsupported.[6]

Extrapolation

Despite its initial goal of extrapolating findings across the original population of 1,606 patients, the government ultimately sought to extrapolate across a broader population. “the findings of [its medical expert] can be applied to the larger patient population comprising of 2,294 patients to arrive at a reasonable estimate of loss.”[7] The expanded population of 2,294 unique Medicare patients (the “revised population”), represented all patients treated by Melgen in the period covered by the indictment from January 2008 to December 2013. The government argued that “limiting the loss calculation to the time period of 2010-2012 would significantly underrepresent the amount of intended loss and actual loss caused by the defendant’s conduct between 2008-2013.”[8]

Rather than articulating a precise loss amount in its filings, the government offered a variety of potential loss calculation methodologies depending on the legal interpretation of “loss” as determined in the sentencing guidelines[9] (i.e., actual loss, intended loss, etc.). The largest loss figures based on its statistical analysis was over $160 million.[10]

Defense Objections
In addition to various legal objections to the government’s loss calculation methodology, Melgen’s counsel also objected to the statistical validity of its extrapolation on several grounds.

Nonrandom Selection of Patients

Counsel argued that the government’s sample of 310 patients was not selected randomly, therefore findings based on that sample could not be validly extrapolated. In particular, the defense took exception with the government’s methodology of selecting multiple samples at random then conducting preliminary analysis on each sample in efforts to select the “best” sample for their purposes. The defense argued that the government effectively “cherry-picked” its sample, which rendered the chosen sample to be effectively nonrandom.[11]

The statistical argument presented here is significant. When discussing randomness in statistics, it is worthwhile to first understand the concept of representativeness. Specifically, “a representative sample contains all of the attributes of the population in the same proportion that they exist in the population. This allows one to generalize from the sample to the population.”[12] If the sample chosen is not reasonably representative of the population, inferences about the population may be irreparably biased and invalid. Further, random sample selection is the method anticipated to lead to a representative sample. Such selection is independent of the sampler, and the resulting sample’s characteristics are anticipated to be predictable based upon of the laws of probability theory. The random selection of a sample removes recognized and unrecognized sources of human bias, such as conscious or unconscious tendencies to select units with larger (or smaller) than average values of the variables of interest (i.e., patients with systematically higher or lower overpayments). Without using proper randomization to select a sample, the presence of selection bias may prevent a sample from being reasonably representative. Notably, randomness also justifies the use of statistical equations to estimate confidence intervals.[13]

While it is prudent to review a randomly selected sample in order to ensure it is reasonably representative of the population, that is not what occurred here. Instead, the government chose multiple random samples simultaneously and compared them to one another, as opposed to selecting one sample and comparing it to the population. In fact, the government admitted that its decision to select sample C ultimately was based on the number of patients diagnosed with macular degeneration in each particular sample, as compared to the other samples.[14] The presence of such decision-making effectively sidesteps the utility of random selection. Said another way, cherry-picking which sample to use, even if all samples were randomly selected, negates its true randomness and injects sources of human bias when determining which patients are ultimately examined.

Sample Not Drawn From the Extrapolated Population

Counsel also objected to the government’s extrapolation of the sample findings across the revised population of patients treated by Melgen from 2008 to 2013, even though the sample was selected from only the original population of patients treated from 2010 to 2012. The difference of almost 700 additional patients increased the government’s affected population — and subsequently its loss estimates — by almost 50 percent. The defense argued that, even if the government’s sample had been drawn randomly, it would only be permissible to extrapolate the findings made about the sample to the population from which it was drawn (i.e., only to the original population).[15]

Again, the statistical objection here is substantial. No matter how a population is defined, the results obtained from random sampling can only be objectively extrapolated to the population from which the sample was drawn. For instance, health care claims sampled from a particular date range, should not be used to yield statistical conclusions about other periods, even at the same facility or treated by the same provider. Such conclusions may be instructive — but they are not statistically objective.

District Court’s Ruling

The court ultimately found flaws in both the government and defense’s statistical analysis and calculated its own loss figures in determining a reasonable loss amount for sentencing.

With regard to defense objections over the randomness of the selected sample, the court found the government’s sampling analysis sufficiently reliable for the purposes of estimating loss in this instance.[16] While it found that the government’s selection and comparison of multiple samples and its subsequent selection of a preferred sample did not follow proper sampling protocols, the court concluded that the government’s actions did not materially misrepresent its analysis. Specifically, the court seemed to conclude that even though the selected sample was not technically random due to the government’s cherry-picking efforts, those actions were not sufficiently malicious or material as to render the sample unrepresentative or invalid. This point was effectively conceded by the defense’s statistical expert, who was cited by the court in its reasoning for allowing the government’s sampling.[17]

However, the court agreed with defense objections regarding the government’s extrapolation across the revised population. The court stated, “Because there was no sampling of patients during the years 2008, 2009 and 2013, the Court will base its findings on the data and statistical analysis relative to the 2010-2012 time period.”[18]

Ultimately, the court calculated its own estimated loss amount by relying on the rates of unsupported billings found in the government’s sample, then extrapolating those rates across total billings for respective procedures from 2010 to 2012. That calculation resulted in an estimated loss amount of $73,417,620.[19] Although the defense was successful in significantly reducing the government’s loss calculation (i.e., by as much as 50 percent), the court overruled the defendant’s objection to the additional 24-level enhancement for loss of more than $65 million but less than $150 million. For purposes of the guideline calculation, the court found that Melgen had an offense level of 38, Criminal History Category I (235-293 months).[20]

Statistical Implications

The presence of cherry-picking can cause an analysis to lose objectivity and become fatally flawed. However, the court in this case found that the degree of bias injected by the government’s actions did not rise to the level of a fatal flaw. For analysts developing and executing sampling analysis, be mindful of methodologically valid procedures for randomization and sample selection, and avoid deviating from these protocols to mitigate potential claims of cherry-picking and nonrandomness. For those scrutinizing opposing analysis, consider objections to both randomness and representativeness. Presenting empirical evidence along with technical arguments may also provide useful evidence when disputing sampling analysis. For instance, in Pruchniewski v. Leavitt, the court found that the plaintiff failed to present any empirical support for his suggestion that a different stratification would have made a material difference in the overpayment calculation or that the method chosen resulted in an unreliable overpayment estimate.[21] Presenting such empirical evidence may have resulted in a more persuasive argument.

As the court found here, there is no basis to objectively extrapolate findings from a sample beyond the population from which the sample was drawn. Analysts should diligently define and document the population for which it seeks to infer findings before selecting the sample. Preliminary evaluation of the population can also help to identify relevant characteristics which can impact the sample design, such as the need for stratification or cluster sampling. When presented with opposing analysis, ensure the sample was validly selected from the population subjected to extrapolation. The nuance and attention-to-detail necessary for this task can often yield errors and inconsistencies, which may ultimately invalidate extrapolated conclusions.

Conclusion

The use of statistical sampling and extrapolation is a growing trend in legal matters and it has become a leading tool in the struggle for efficiency when collecting evidence. This case illustrates how technical flaws in sampling analysis might be insufficient to invalidate the underlying conclusions. On the other hand, it also highlights the significant benefits of diligent scrutiny by opposing parties. Whether scrutinizing opposing analysis or developing your own, a thorough understanding of statistical sampling continues to be an asset in government enforcement.

The article was co-authored by Forensus Managing  Director Chris Haney. It was originally published in Law360 Expert Analysis.

[1] Dkt. No. 405, Gov’t Response to Objections to Presentence Investigation Report, at 38.

[2] Dkt. 489, Judgment.

[3] Dkt. 492, Def.’s Mot. for Bond Pending Appeal and Motion for New Trial.

[4] Dkt. 493, Notice of Appeal

[5] Dkt. No. 405, Gov’t Response to Objections to Presentence Investigation Report, at 38.

[6] Id.

[7] Id. at 41

[8] Id.

[9] Pursuant to U.S. Sentencing Guideline § 2B1.1., in cases involving a Government health care program, such as Medicare, the aggregate amount of fraudulent bills submitted to the Government health care program shall constitute prima facie evidence of the amount of the intended loss, unless rebutted.

[10] Dkt. 405-1, Gov’t Loss Calculations Spreadsheet

[11] Dkt. 400, Def.’s Objections to Presentence Investigation Report at 30.

[12] Eric W. Corty, Using and Interpreting Statistics (2007), p145.

[13] Reference Manual on Scientific Evidence, p230.

[14] Dkt. No. 405, Gov’t Response to Objections to Presentence Investigation Report, at 38.

[15] Dkt. 400, Def.’s Objections to Presentence Investigation Report at 33

[16] Dkt. 486, Order at 3.

[17] Id. at 4.

[18] Id. at 3.

[19] Id. at 5.

[20] Id.

[21] Pruchniewski v. Leavitt, Case No. 8:04-CV-2200, 2006 WL 2331071 at *12 (M.D. Fla Aug. 10, 2006).