Abstract

An example of methylation analysis with Methyl-IT, a novel methylome analysis procedure based on a signal-detection and machine- learning approach, is provided here. Methylation analysis involves a signal detection problem, and the method was designed to discriminate methylation regulatory signal from background noise induced by thermal fluctuations. Methyl-IT enhances the resolution of genome methylation behavior to reveal network-associated responses, offering resolution of gene pathway influences not attainable with previous methods. Herein, an example of MethylIT application to the analysis of breast cancer methylomes is presented.

# Methyl-IT

MethylIT is an R package for methylome analysis based on a signal-detection machine learning approach (SD-ML). This approach is postulated to provide greater sensitivity for resolving true signal from the methylation background within the methylome (Sanchez and Mackenzie 2016; Sanchez et al. 2019). Because the biological signal created within the dynamic methylome environment characteristic of plants is not free from background noise, the approach, designated Methyl-IT, includes the application of signal detection theory (Greiner, Pfeiffer, and Smith 2000; Carter et al. 2016; Harpaz et al. 2013; Kruspe et al. 2017).

A basic requirement for the application of signal detection is the knowledge of the probability distribution of the background noise, which is used as null hypothesis in the detection of the methylation signal. This probability distribution is, in general, a member of generalized gamma distribution family, and it can be deduced on a statistical mechanical/thermodynamics basis from DNA methylation induced by thermal fluctuations (Sanchez and Mackenzie 2016). Herein, we provide an example of Methyl-IT application to the analysis of breast cancer methylomes. Due to the size of human methylome the current example only covers the analysis of chromosome 13. A full description of Methyl-IT application of methylome analysis in plants is given in the manuscript (Sanchez et al. 2019).

The R packages required in this example are:

suppressPackageStartupMessages({
library(MethylIT)
library(ggplot2) # graphic
library(reshape2) # To reshape the data frame
library(grid) # For multiple plots
library(gridExtra) # For multiple plots
library(VennDiagram)
library(rtracklayer) # To import gene annotation
}
)

## Available datasets and reading

Methylome datasets of whole-genome bisulfite sequencing (WGBS) are available at Gene Expression Omnibus (GEO DataSets). For the current example, datasets from breast tissues (normal and cancer) and embryonic stem cells will be downloaded from GEO. The data set are downloaded providing the GEO accession numbers for each data set to the function ‘getGEOSuppFiles’ (see the function getGEOSuppFiles help). Some times the ftp site is not available and the information must be manually downloaded from GEO database.

# Embryonic stem cells datasets
esc.files = getGEOSuppFiles(GEO = c("GSM2041690", "GSM2041691", "GSM2041692"),
verbose = FALSE)
# Breast tissues (normal, cancer, metastasis)
cancer.files = getGEOSuppFiles(GEO = c("GSM1279517","GSM1279514",
"GSM1279513"), verbose = FALSE)

The file path and name of each downloaded dataset is found in the output variables ‘esc.files’ and ‘cancer.files’.

Datasets for our example can be read with function readCounts2GRangesList. To specify the reading of only chromosome 13, we can specify the parameter ‘chromosomes = “Chr13”’. The symbol chromosome 13, in this case “Chr13”, must be consistent with the annotation provided in the given GEO dataset. Each file is wholly read with the setting ‘chromosomes = “Chr13”’ and then the GRanges are built only with chromosome 13, which could be time consuming. However, users working on Linux OS can specify the reading of specific lines from each file by using regular expressions. For example, if only chromosomes 1 and 3 are required, then we can set chromosomes = NULL (default) and ‘chromosome.pattern = “^Chr[1,3]”’. This will read all the lines in the downloaded files starting with the words “Chr1” or “Chr3”. If we are interested in chromosomes 1 and 2, then we can set ‘chromosome.pattern = “^Chr[1-2]”’. If all the chromosomes are required, then set chromosomes = NULL and chromosome.pattern = NULL (default). Obviously, before read the files, user must check them to see which annotation was used by the experimenters to denote the chromosomes, e.g., “chr13” or just “13”, etc.

# Embryonic stem cells datasets
ref = readCounts2GRangesList(filenames = esc.files,
sample.id = c("ESC1","ESC2", "ESC3"),
columns = c(seqnames = 1, start = 2,
mC = 4, uC = 5), pattern = "^chr13",
remove = TRUE, verbose = FALSE)
# Breast tissues (normal, cancer, metastasis)
LR = readCounts2GRangesList(filenames = cancer.files,
sample.id = c("Breast_normal", "Breast_cancer",
"Breast_metastasis"),
columns = c(seqnames = 1, start = 2,
mC = 3, uC = 4),
remove = TRUE, pattern = "^13",
chromosome.names = "chr13", verbose = FALSE)

In real data analysis, a bigger sample size must be used. For the purpose of this example, to illustrate the application of methylation analysis with Methyl-IT, we will just add an artificial normal sample.

LR$Breast_normal1 <- poolFromGRlist(c(ref[1], LR$Breast_normal,
LR$Breast_normal, LR$Breast_normal,
LR$Breast_normal, LR$Breast_normal),
stat = "mean", num.cores = 6L,
verbose = FALSE)

In the metacolumn of the last GRanges object, mC and uC stand for the methylated and unmethylated read counts, respectively. Notice that option ‘remove = TRUE’ remove the decompressed files (default: FALSE, see ? readCounts2GRangesList for more details about this function).

## The reference individual

Any two objects can be compared based on some measured variable if, and only if, the measurements where taken in the same metric space, the same coordinate system, in respect to the same origin of coordinates. Usually, in our daily 3D experience, our brain automatically sets up the origin of coordinates equal to zero. The differences found in the comparison depend on the reference used to perform the measurements and from the metric system. The space where the objects are located (or the set of objects) together with the metric is called metric space.

Unfortunately, this is not the case for the methylation process. Each individual from the same population and even each chromosome follows an independent stochastic methylation process. This is not a surprise, since cytosine methylation affect the mechanical properties of DNA molecule with strong dependence of the DNA sequence context (until, at least. 6 bases at both sides of each cytosine site) (Severin et al. 2011; Ngo et al. 2016).

To evaluate the methylation differences between individuals from control and treatment we introduce a metric in the bidimensional space of methylation levels $$P_i = (p_i, 1-p_i)$$. Vectors $$P_i$$ provide a measurement of the uncertainty of methylation levels (Sanchez et al. 2019). However, to perform the comparison between the uncertainty of methylation levels from each group of individuals, control $$(c)$$ and treatment $$(t)$$, we should estimate the uncertainty variation with respect to the same individual reference on the mentioned metric space. The reason to measure the uncertainty variation with respect to the same reference resides in that even sibling individuals follow an independent ontogenetic development and, consequently, their developments follow independent stochastic processes. This a consequence of the “omnipresent” action of the second law of thermodynamics in living organisms. In the current example, we will create the reference individual by pooling the methylation counts from the embryonic stem cells.

It should be noticed that the results are sensitive to the reference used. The statistics mean, median, or sum of the read counts at each cytosine site of some control samples can be used to create a virtual reference sample. A virtual reference individual can be built with function poolFromGRlist. It is up to the user whether to apply the ‘row sum’, ‘row mean’ or ‘row median’ of methylated and unmethylated read counts at each cytosine site across individuals. Notice that when “mean” is selected the virtual reference individual is the group centroid, which play a fundamental role in multivariate statistic.

Ref = poolFromGRlist(ref, stat = "mean", num.cores = 6L, verbose = FALSE)

Ref
## GRanges object with 1560637 ranges and 2 metadata columns:
##             seqnames    ranges strand |        mC        uC
##                <Rle> <IRanges>  <Rle> | <numeric> <numeric>
##         [1]    chr13  19020631      * |         1         1
##         [2]    chr13  19020633      * |         2         2
##         [3]    chr13  19020642      * |         1         1
##         [4]    chr13  19020643      * |         2         2
##         [5]    chr13  19020679      * |         1         1
##         ...      ...       ...    ... .       ...       ...
##   [1560633]    chr13 115108993      * |         1         3
##   [1560634]    chr13 115109022      * |         1         1
##   [1560635]    chr13 115109023      * |         3         4
##   [1560636]    chr13 115109523      * |         2         2
##   [1560637]    chr13 115109524      * |         1         1
##   -------
##   seqinfo: 1 sequence from an unspecified genome; no seqlengths

Only direct lab experiments can reveal whether differences detected with distinct references outside the experimental conditions for control and treatment groups are real. The best reference would be estimated using a subset of individuals from control group. Such a reference will contribute to remove the intragroup variation, in control and in treatment groups, induced by environmental changes external to or not controlled by the experimental conditions.

Methylation analysis for each cytosine position is frequently performed in the bidimensional space of $$(methylated, unmethylated)$$ read counts. Frequently, Fisher’s exact test is applied to a single cytosine position, under the null hypothesis that the proportions $$p_{ct}=methylated_{ct}/(methylated_{ct}+unmethylated_{ct})$$ and $$p_{tt}=methylated_{tt}/(methylated_{tt}+unmethylated_{tt})$$ are the same for control and treatment, respectively. In this case, the implicit reference point for the counts at every cytosine positions is $$(methylated = 0,unmethylated = 0)$$, which corresponds to the point $$P_i = (0, 1)$$.

In our case, the Hellinger divergence (one of the metric used, here) of each individual in respect to the reference is the variable to test in place of $$(methylated, unmethylated)$$ read counts or the methylation levels $$P_i = (p_i, 1-p_i)$$.

The use of references is restricted by the thermodynamics basis of the theory. The current information-thermodynamics based approach is supported on the following postulate: “High changes of Hellinger divergences are less frequent than low changes, provided that the divergence is proportional to the amount of energy required to process the change of information in the methylation system”.

The last postulate acknowledges the action of the second law of thermodynamics on the biomolecular methylation system. For the methylation system, it implies that the frequencies of the information divergences between methylation levels must be proportional to a Boltzmann factor (see supplementary information from reference (Sanchez and Mackenzie 2016)). In other words, the frequencies of information divergences values should follow a trend proportional to an exponential decay.

If we do not observe such a behavior, then either the reference is too far from experimental condition or we are dealing with an extreme situation where the methylation machinery in the cell is dysfunctional. The last situation is found, for example, in the silencing mutation at the gene of cytosine-DNA-methyltransferase in Arabidopsis thaliana. Methylation of 5-methylcytosine at CpG dinucleotides is maintained by MET1 in plants.

## Hellinger divergence estimation

To perform the comparison between the uncertainty of methylation levels from each group of individuals, control $$(c)$$ and treatment $(t)$, the divergence between the methylation levels of each individual is estimated with respect to the same reference on the metric space formed by the vector set $$P_i = (p_i, 1-p_i)$$ and the Hellinger divergence $$H$$. Basically, the information divergence between the methylation levels of an individual $$j$$ and reference sample $$r$$ is estimated according to the Hellinger divergence given by the formula:

$H(\hat p_{ij},\hat p_{ir}) = w_i[(\sqrt{\hat p_{ij}} - \sqrt{\hat p_{ir}})^2+(\sqrt{1-\hat p_{ij}} - \sqrt{1-\hat p_{ir}})^2]$ where $w_i = 2$, $$m_{ij} = {n_i}^{mC_j}+{n_i}^{uC_j}+1$$, $$m_{ir} = {n_i}^{mC_r}+ {n_i}^{uC_r}+1$$ and $$j \in {\{c,t}\}$$.

This equation for Hellinger divergence is given in reference (Basu, Mandal, and Pardo 2010), but other information theoretical divergences can be used as well. Next, the information divergence for control (Breast_normal) and treatment (Breast_cancer and Breast_metastasis) samples are estimated with respect to the reference virtual individual. A Bayesian correction of counts can be selected or not. In a Bayesian framework, methylated read counts are modeled by a beta-binomial distribution, which accounts for both the biological and sampling variations (Hebestreit, Dugas, and Klein 2013; Robinson et al. 2014; Dolzhenko and Smith 2014).

In our case we adopted the Bayesian approach suggested in reference (Baldi and Brunak 2001) (Chapter 3). In a Bayesian framework with uniform priors, the methylation level can be defined as: $$p = (mC + 1)/ (mC + uC + 2)$$. However, the most natural statistical model for replicated BS-seq DNA methylation measurements is beta-binomial (the beta distribution is a prior conjugate of binomial distribution). We consider the parameter $$p$$ (methylation level) in the binomial distribution as randomly drawn from a beta distribution. The hyper-parameters $$\alpha$$ and $$\beta$$ from the beta-binomial distribution are interpreted as pseudo-counts. The information divergence is estimated here using the function estimateDivergence:

HD = estimateDivergence(ref = Ref, indiv = LR, Bayesian = TRUE,
min.coverage = 5, high.coverage = 300,
percentile = 0.999, num.cores = 6L, tasks = 0L,
verbose = FALSE)

HD$Breast_cancer ## GRanges object with 584538 ranges and 9 metadata columns: ## seqnames ranges strand | c1 t1 c2 t2 ## <Rle> <IRanges> <Rle> | <numeric> <numeric> <numeric> <numeric> ## [1] chr13 19020631 * | 1 1 14 24 ## [2] chr13 19020633 * | 2 2 14 25 ## [3] chr13 19020643 * | 2 2 7 38 ## [4] chr13 19020782 * | 1 1 6 56 ## [5] chr13 19020786 * | 1 1 10 53 ## ... ... ... ... . ... ... ... ... ## [584534] chr13 115108776 * | 1 1 52 20 ## [584535] chr13 115108789 * | 2 2 27 43 ## [584536] chr13 115108993 * | 1 3 72 5 ## [584537] chr13 115109023 * | 3 4 56 36 ## [584538] chr13 115109524 * | 1 1 31 9 ## p1 p2 TV ## <numeric> <numeric> <numeric> ## [1] 0.454104856214051 0.390938141872359 -0.131578947368421 ## [2] 0.45434070807688 0.381536829997737 -0.141025641025641 ## [3] 0.45434070807688 0.186316886276701 -0.344444444444444 ## [4] 0.454104856214051 0.121789629914384 -0.403225806451613 ## [5] 0.454104856214051 0.180923689744828 -0.341269841269841 ## ... ... ... ... ## [584534] 0.454104856214051 0.722219851021173 0.222222222222222 ## [584535] 0.45434070807688 0.397688717430985 -0.114285714285714 ## [584536] 0.451771243822349 0.928153611792893 0.685064935064935 ## [584537] 0.453415070880135 0.611794539917136 0.180124223602485 ## [584538] 0.454104856214051 0.771794032169105 0.275 ## bay.TV hdiv ## <numeric> <numeric> ## [1] -0.0631667143416917 0.0228027372314754 ## [2] -0.0728038780791427 0.0484924086963633 ## [3] -0.268023821800179 0.770678496718866 ## [4] -0.332315226299667 0.829702372928104 ## [5] -0.273181166469223 0.512263189402768 ## ... ... ... ## [584534] 0.268114994807123 0.436970408874593 ## [584535] -0.0566519906458947 0.0306816345240809 ## [584536] 0.476382367970543 2.89429516403061 ## [584537] 0.158379469037002 0.373542098318946 ## [584538] 0.317689175955054 0.615406476163056 ## ------- ## seqinfo: 1 sequence from an unspecified genome; no seqlengths Function ‘estimateDivergence’ returns a list of GRanges objects with the four columns of counts, the information divergence, and additional columns: 1. The original matrix of methylated ($c_i$) and unmethylated ($t_i$) read counts from control ($i=1$) and treatment ($i=2$) samples. 2. "p1" and "p2": methylation levels for control and treatment, respectively. 3. "bay.TV": total variation TV = p2 - p1. 4. "TV": total variation based on simple counts:$TV=c1/(c1+t1)-
c2/(c2+t2)$. 5. "hdiv": Hellinger divergence. If Bayesian = TRUE, results are based on the posterior estimations of methylation levels $$p1$$ and $$p2$$. Filtering by coverage is provided at this step which would be used unless previous filtering by coverage had been applied. This is a pairwise filtering. Cytosine sites with ‘coverage’ > ‘min.coverage’ and ‘coverage’ < ‘percentile’ (e.g., 99.9 coverage percentile) in at least one of the samples are preserved. The coverage percentile used is the maximum estimated from both samples: reference and individual. For some GEO datasets only the methylation levels for each cytosine site are provided. In this case, Hellinger divergence can be estimated as given in reference (Sanchez and Mackenzie 2016): $H(\hat p_{ij},\hat p_{ir}) = 2 \Big[(\sqrt{\hat p_{ij}} - \sqrt{\hat p_{ir}})^2+(\sqrt{1-\hat p_{ij}} - \sqrt{1-\hat p_{ir}})^2\Big]$ ## Descriptive statistical analysis A descriptive analysis on the distribution of Hellinger divergences of methylation levels is recommended. Like in any other statistical analysis, descriptive statistical analysis help us to detect potential issues in the raw data. It is the user responsibility to perform quality check of his/her dataset before start to apply Methyl-IT. Nevertheless, the quality checking of the raw data is not perfect. So, it is healthy to sees for potential issues. ### Histogram and boxplots of divergences estimated in each sample First, the data of interest (Hellinger divergences, “hdiv”) are selected from the GRanges objects: normal = HD$Breast_normal[, "hdiv"]
normal1 = HD$Breast_normal1[, "hdiv"] cancer = HD$Breast_cancer[, "hdiv"]
metastasis = HD$Breast_metastasis[, "hdiv"] Next, a single GRanges object is built from the above set of GRanges objects using the function uniqueGRanges. Notice that the number of cores to use for parallel computation can be specified. hd = uniqueGRanges(list(normal, normal1, cancer, metastasis), missing = NA, verbose = FALSE, num.cores = 6L) colnames(mcols(hd)) <- c("normal", "normal1", "cancer", "metastasis") hd ## GRanges object with 793551 ranges and 4 metadata columns: ## seqnames ranges strand | normal normal1 ## <Rle> <IRanges> <Rle> | <numeric> <numeric> ## [1] chr13 19020631 * | 0.288068219746143 0.28767862804324 ## [2] chr13 19020633 * | 0.869230665921108 0.714188285613248 ## [3] chr13 19020643 * | 0.555862275706214 0.560689033273912 ## [4] chr13 19020680 * | 0.023720232658528 0.0281645245579912 ## [5] chr13 19020687 * | 7.79397480586338e-05 0.00200990552411886 ## ... ... ... ... . ... ... ## [793547] chr13 115108776 * | 0.575490027087112 0.571096253172782 ## [793548] chr13 115108789 * | 0.519826661201187 0.491408506478641 ## [793549] chr13 115108993 * | 2.52414786540235 2.25137133006846 ## [793550] chr13 115109023 * | 3.03241003774626 2.95959362743878 ## [793551] chr13 115109524 * | 0.946527350542327 0.920987828967544 ## cancer metastasis ## <numeric> <numeric> ## [1] 0.0228027372314754 2.07603549250268 ## [2] 0.0484924086963633 3.1812220976813 ## [3] 0.770678496718866 0.000733475976828028 ## [4] <NA> <NA> ## [5] <NA> <NA> ## ... ... ... ## [793547] 0.436970408874593 <NA> ## [793548] 0.0306816345240809 <NA> ## [793549] 2.89429516403061 3.28533077952636 ## [793550] 0.373542098318946 0.671734167824807 ## [793551] 0.615406476163056 <NA> ## ------- ## seqinfo: 1 sequence from an unspecified genome; no seqlengths Now, the Hellinger divergences estimated for each sample are in a single matrix on the metacolumn of the GRanges object and we can proceed to build the histogram and boxplot graphics for these data. Finally, the data included in the graphic are: # Define an auxiliar function fun_length <- function(x){ return(data.frame(y = median(x) + 1, label = paste0("n = ", length(x)))) } data <- data.frame(normal = hd$normal, normal1 = hd$normal1, cancer = hd$cancer, metastasis = hd$metastasis) data = suppressMessages(melt(data)) colnames(data) <- c("Breast.tissue", "HD") data = data[data$HD > 0, ]
DataFrame(data)
## DataFrame with 3174204 rows and 2 columns
##           Breast.tissue                   HD
##                <factor>            <numeric>
## 1                normal    0.288068219746143
## 2                normal    0.869230665921108
## 3                normal    0.555862275706214
## 4                normal    0.023720232658528
## 5                normal 7.79397480586338e-05
## ...                 ...                  ...
## NA.617427            NA                   NA
## NA.617428            NA                   NA
## 3174202      metastasis     3.28533077952636
## 3174203      metastasis    0.671734167824807
## NA.617429            NA                   NA
p1 = ggplot(data, aes(x = HD, fill = Breast.tissue, colour = Breast.tissue)) +
geom_histogram(alpha = 0.5, binwidth = 0.5, position = "identity",
na.rm = TRUE, size = 0.7) + xlim(-0.4,20) +
theme(axis.title.x = element_text(face = "bold", size = 20),
axis.text.x = element_text(face = "bold", size = 20,
color =  "black", hjust = 0.5,
vjust = 0.75),
axis.text.y = element_text(face = "bold", size = 20, color = "black"),
axis.title.y = element_text(face = "bold", size = 20,color = "black"),
legend.text = element_text(size = 20, face = "bold"),
legend.title = element_text(size = 20, face = "bold")) +
ylab( "Counts" ) +
ggtitle("Histogram of Hellinger Divergence")

# For visualization purposes HD is limited to the interval 0 to 50
tv.95 = quantile(abs(x$bay.TV), 0.95) return(c(tv = tv.95, hd = hd.95)) })) critical.val ## tv.95% hd.95% ## Breast_normal 0.5044770 3.787033 ## Breast_cancer 0.5186583 4.385264 ## Breast_metastasis 0.5106559 4.030725 ## Breast_normal1 0.4903243 3.278097 ## Nonlinear fit of probability distribution models A basic requirement for the application of signal detection is the knowledge of the probability distribution of the background noise. Probability distribution, as a Weibull distribution model, can be deduced on a statistical mechanical/thermodynamics basis for DNA methylation induced by thermal fluctuations (Sanchez and Mackenzie 2016). Assuming that this background methylation variation is consistent with a Poisson process, it can be distinguished from variation associated with methylation regulatory machinery, which is non-independent for all genomic regions (Sanchez and Mackenzie 2016). An information-theoretic divergence to express the variation in methylation induced by background thermal fluctuations will follow a Weibull distribution model, provided that it is proportional to the minimum energy dissipated per bit of information associated with the methylation change. The nonlinear fit to a Weibull distribution model is performed by the function nonlinearFitDist. In general, an information divergence of methylation levels will follow a probability distribution from the family of generalized gamma distribution. Next, the best fitted model will be selected from one of the following distributions: two- and three-parameter Weibull distributions, gamma with two- and three-parameters (“Gamma2P” or “Gamma3P”), and generalized gamma with three-parameter (“GGamma3P”). The best model is selected on the basis of Akaike’s information criterion and the correlation coefficient of cross-validations for the nonlinear regressions (R.Cross.val). These criteria evaluate different information inferred from the models. AIC deals with the trade-off between the goodness of fit (GOF) of the model and the complexity of the model, while R.Cross.val provides information on the prediction power/performance of the model when confronted with external dataset. Cross-validations for the nonlinear regressions (R.Cross.val) were performed as described in reference (Stevens 2009). In addition, Stein’s formula for adjusted R squared ($$\rho$$) was used as an estimator of the average cross-validation predictive power (Stevens 2009). In general, the best fitted model yields the best DMP classification performance (see below). However, on special dataset, as the used in the current example. Function gofReport search for the best fitted model between the set of models requested by the user. The best model including “GGamma3P” model: d <- c("Weibull2P", "Weibull3P", "Gamma2P", "Gamma3P", "GGamma3P") gof_1 <- gofReport(HD = HD, column = 9L, model = d, num.cores = 6L, output = "all", verbose = FALSE) ## | | | 0% | |============== | 20% | |============================ | 40% | |========================================== | 60% | |======================================================== | 80% | |======================================================================| 100% ## ## *** Creating report ... gof_1$bestModel
##     Breast_normal     Breast_cancer Breast_metastasis    Breast_normal1
##        "GGamma3P"        "GGamma3P"        "GGamma3P"        "GGamma3P"

The best model excluding “GGamma3P” model:

d <- c("Weibull2P", "Weibull3P", "Gamma2P", "Gamma3P")
gof_2 <- gofReport(HD = HD, column = 9L, model = d, num.cores = 6L,
output = "all", verbose = FALSE)
##
|
|                                                                      |   0%
|
|==================                                                    |  25%
|
|===================================                                   |  50%
|
|====================================================                  |  75%
|
|======================================================================| 100%
##
##  *** Creating report ...
gof_2$bestModel ## Breast_normal Breast_cancer Breast_metastasis Breast_normal1 ## "Weibull2P" "Gamma2P" "Gamma2P" "Weibull2P" When “GGamma3P” model is included, the goodness-of-fit indicators used here suggest that it is best fitted model. However, a final decision must take into account the DMP classification performance as well (see below), which is accomplished after a signal detection step. The reader must keep in mind that we are dealing here with numerical algorithms, which are not perfect. Depending on dataset, the numerical computation will confront different challengers. The difference between g3p_R.Cross.val and gg3p_R.Cross.val suggests that model predictions powers for “GGamma3P” and “Gamma3P” are close. gof_1$stats
##                    w2p_AIC w2p_R.Cross.val w3p_AIC w3p_R.Cross.val  g2p_AIC
## Breast_normal     -3083858       0.9950587      NA              NA -3000096
## Breast_cancer     -1846819       0.9878989      NA              NA -1967840
## Breast_metastasis -1465762       0.9871201      NA              NA -1643581
## Breast_normal1    -3630240       0.9974563      NA              NA -3478122
##                   g2p_R.Cross.val g3p_AIC g3p_R.Cross.val gg3p_AIC
## Breast_normal           0.9932869     Inf               0 -4102368
## Breast_cancer           0.9883060     Inf               0 -2597724
## Breast_metastasis       0.9906769     Inf               0 -2150421
## Breast_normal1          0.9967128     Inf               0 -5253037
##                   gg3p_R.Cross.val bestModel
## Breast_normal            0.9983766      gg3p
## Breast_cancer            0.9959062      gg3p
## Breast_metastasis        0.9969623      gg3p
## Breast_normal1           0.9996704      gg3p

It is worthy to observe that although in the current example we successfully performed the nonlinear fit for GGamma3P model, this is not the general case when the number of cytosine sites goes close or above $$10^6$$, for which the computational cost is very high. The nonlinear fit of GGamma3P or GGamma4P is, in general, difficult since two different set of estimated parameters could produce very close distribution curves. Also, it is important to check the value of the scaling parameter from each distribution. The numerical algorithms could find the best fitted model with scaling parameter values close to zero, say e.g. $$10^{-8}$$, and with the highest GOFs. Situations like this are biologically meaningless and the fitted models must be discarded (Sanchez and Mackenzie 2016).

## Signal detection

The information thermodynamics-based approach is postulated to provide greater sensitivity for resolving true signal from the thermodynamic background within the methylome (Sanchez and Mackenzie 2016). Since the biological signal created within the dynamic methylome environment characteristic of living organisms is not free from background noise, the approach, designated Methyl-IT, includes the application of signal detection theory (Greiner, Pfeiffer, and Smith 2000; Carter et al. 2016; Harpaz et al. 2013; Kruspe et al. 2017). Signal detection is a critical step to increase sensitivity and resolution of methylation signal by reducing the signal-to-noise ratio and objectively controlling the false positive rate and prediction accuracy/risk.

## Potential methylation signal

The first estimation in our signal detection step is the identification of the cytosine sites carrying potential methylation signal $$PS$$. The methylation regulatory signal does not hold the theoretical distribution and, consequently, for a given level of significance $$\alpha$$ (Type I error probability, e.g. $$\alpha = 0.05$$), cytosine positions $$k$$ with information divergence $$H_k >= H_{\alpha = 0.05}$$ can be selected as sites carrying potential signals $$PS$$. The value of $$\alpha$$ can be specified.

For example, potential signals with $$H_k > H_{\alpha = 0.01}$$ can be selected. For each sample, cytosine sites are selected based on the corresponding fitted theoretical distribution model estimated in the previous step. Additionally, since cytosine with $$|TV_{d_k}| < 0.1$$ are the most abundant sites, depending on the sample (experiment), cytosine positions $$k$$ with $$H_k >= H_{\alpha = 0.05}$$ and $$|TV_{d_k}| < 0.1$$ can be observed. To prevent the last situation we can select the $$PS$$ with the additional constraint $$|TV_{d_k}| > TV_0$$, where $$TV_0$$ (‘tv.cut’) is a user specified value. The $$PS$$ is detected with the function getPotentialDIMP:

PS1 = getPotentialDIMP(LR = HD, dist.name = gof_1$bestModel, nlms = gof_1$nlms,
div.col = 9, alpha = 0.05, tv.col = 8, tv.cut = 0.5)
PS2 = getPotentialDIMP(LR = HD, dist.name = gof_2$bestModel, nlms = gof_2$nlms,
div.col = 9, alpha = 0.05, tv.col = 8, tv.cut = 0.5)
PS1$Breast_normal ## GRanges object with 20170 ranges and 10 metadata columns: ## seqnames ranges strand | c1 t1 c2 t2 ## <Rle> <IRanges> <Rle> | <numeric> <numeric> <numeric> <numeric> ## [1] chr13 19082356 * | 2 3 32 0 ## [2] chr13 19177615 * | 4 4 23 0 ## [3] chr13 19240007 * | 2 2 47 0 ## [4] chr13 19240932 * | 3 3 32 0 ## [5] chr13 19296278 * | 3 3 28 0 ## ... ... ... ... . ... ... ... ... ## [20166] chr13 115096223 * | 2 2 44 0 ## [20167] chr13 115096666 * | 3 3 34 0 ## [20168] chr13 115097021 * | 3 3 21 0 ## [20169] chr13 115097976 * | 2 2 42 0 ## [20170] chr13 115098695 * | 5 6 27 0 ## p1 p2 TV ## <numeric> <numeric> <numeric> ## [1] 0.451273311018461 0.966917401618622 0.6 ## [2] 0.45304420669874 0.956663168622331 0.5 ## [3] 0.452523760640444 0.976274020307531 0.5 ## [4] 0.45278541784448 0.966917401618622 0.5 ## [5] 0.45278541784448 0.96302946432176 0.5 ## ... ... ... ... ## [20166] 0.452523760640444 0.974851489843536 0.5 ## [20167] 0.45278541784448 0.968570038217104 0.5 ## [20168] 0.45278541784448 0.953457324516072 0.5 ## [20169] 0.452523760640444 0.973804423693521 0.5 ## [20170] 0.452068007958716 0.961910370769917 0.545454545454545 ## bay.TV hdiv wprob ## <numeric> <numeric> <numeric> ## [1] 0.51564409060016 4.15703316933603 0.0123894226716491 ## [2] 0.50361896192359 4.91437397287391 0.000710758152019684 ## [3] 0.523750259667087 4.0095064341879 0.018433872546532 ## [4] 0.514131983774142 4.70737325955354 0.00182094549570408 ## [5] 0.51024404647728 4.45308383566372 0.00485450539495913 ## ... ... ... ... ## [20166] 0.522327729203092 3.9325618899715 0.0223160365851342 ## [20167] 0.515784620372624 4.82115443364787 0.0011045175272428 ## [20168] 0.500671906671593 3.89487903785825 0.0244132962250764 ## [20169] 0.521280663053077 3.87742927411522 0.0254294742316257 ## [20170] 0.509842362811201 6.58905765063053 1.85491151602445e-10 ## ------- ## seqinfo: 1 sequence from an unspecified genome; no seqlengths PS2$Breast_normal
## GRanges object with 8919 ranges and 10 metadata columns:
##          seqnames    ranges strand |        c1        t1        c2        t2
##             <Rle> <IRanges>  <Rle> | <numeric> <numeric> <numeric> <numeric>
##      [1]    chr13  19177615      * |         4         4        23         0
##      [2]    chr13  19240932      * |         3         3        32         0
##      [3]    chr13  19366559      * |         3         3        41         0
##      [4]    chr13  19377594      * |         3         5        40         1
##      [5]    chr13  19403993      * |         4         4        26         0
##      ...      ...       ...    ... .       ...       ...       ...       ...
##   [8915]    chr13 115090609      * |         4         4        60         1
##   [8916]    chr13 115090684      * |         4         4        76         2
##   [8917]    chr13 115094605      * |         3         3        38         0
##   [8918]    chr13 115096666      * |         3         3        34         0
##   [8919]    chr13 115098695      * |         5         6        27         0
##                         p1                p2                TV
##                  <numeric>         <numeric>         <numeric>
##      [1]  0.45304420669874 0.956663168622331               0.5
##      [2]  0.45278541784448 0.966917401618622               0.5
##      [3]  0.45278541784448 0.973247499292334               0.5
##      [4] 0.450303646023985 0.951987253109134 0.600609756097561
##      [5]  0.45304420669874 0.960721412782865               0.5
##      ...               ...               ...               ...
##   [8915]  0.45304420669874 0.966311688871077 0.483606557377049
##   [8916]  0.45304420669874 0.961226987328427 0.474358974358974
##   [8917]  0.45278541784448 0.971424962282638               0.5
##   [8918]  0.45278541784448 0.968570038217104               0.5
##   [8919] 0.452068007958716 0.961910370769917 0.545454545454545
##                     bay.TV             hdiv              wprob
##                  <numeric>        <numeric>          <numeric>
##      [1]  0.50361896192359 4.91437397287391 0.0391864397601092
##      [2] 0.514131983774142 4.70737325955354 0.0453083217221566
##      [3] 0.520462081447855  5.1642065228043 0.0328714151704236
##      [4] 0.501683607085148 5.41958287747074 0.0274509788905275
##      [5] 0.507677206084125 5.22970386210739 0.0313884759937581
##      ...               ...              ...                ...
##   [8915] 0.513267482172336 6.36926774422415 0.0139793042162816
##   [8916] 0.508182780629686 6.28477237221604 0.0148485285739722
##   [8917] 0.518639544438159 5.02660979748285  0.036214397027718
##   [8918] 0.515784620372624 4.82115443364787 0.0418357714100688
##   [8919] 0.509842362811201 6.58905765063053 0.0119463110012273
##   -------
##   seqinfo: 1 sequence from an unspecified genome; no seqlengths

Model “GGamma3P” reported 20263 - 11314 = 8949 more DMPs in the normal tissue than model “Gamma3P”.

Notice that the total variation distance $$TV_d=|TV|$$ is an information divergence as well and that it can be used in place of Hellinger divergence (Sanchez and Mackenzie 2016). The set of vectors $$P_i = (p_i, 1-p_i)$$ and distance function $$TV_d$$ integrate a metric space (Sanchez et al. 2019). In particular, after takes the Manhattan distance between healthy individuals and patients methylation levels we have:

$M_d(p_{ij}, p_{ir})=\frac{1}{2}(|\hat p_{ij} - \hat p_{ir}| + | (1-\hat p_{ij}) - (1-\hat p_{ir})|)=|\hat p_{ij} - \hat p_{ir}|=TV_d(p_{ij}, p_{ir})$.

That is, the absolute difference of methylation levels is a particular case of Manhattan distance, which is named total variation distance $$TV_d$$. The quantitative effect of the vector components $$1-\hat p_{ij}$$ and $$1-\hat p_{ir}$$ (in our case, the effect of unmethylated read counts) is not present in $$TV_d$$ as in $$H(\hat p_{ij},\hat p_{ir})$$.

## Histogram and boxplots of methylation potential signals

As before, a single GRanges object is built from the above set GRanges objects using the function uniqueGRanges, and the Hellinger divergences of the cytosine sites carrying $$PS$$ (for each sample) are located in a single matrix on the metacolumn of the GRanges object.

# Define an auxiliar function
fun_length <- function(x){
return(data.frame(y = median(x) + 1.5, label = paste0("n = ", length(x))))
}

# The data to be used in the boxplot
ps1 = uniqueGRanges(PS1, missing = NA, columns = 9, verbose = FALSE,
num.cores = 6L)
colnames(mcols(ps1)) <- c("normal", "cancer", "metastasis", "normal1")
dat1 = data.frame(normal = ps1$normal, normal1 = ps1$normal1,
cancer = ps1$cancer, metastasis = ps1$metastasis)
dat1 = suppressMessages(melt(dat1))
colnames(dat1) <- c("Breast.tissue", "HD")
idx <- which(is.na(dat1$HD)) dat1 <- dat1[-idx, ] # To remove missing data ps2 <- uniqueGRanges(PS2, missing = NA, columns = 9, verbose = FALSE, num.cores = 6L) colnames(mcols(ps2)) <- c("normal", "cancer", "metastasis", "normal1") dat2 = data.frame(normal = ps2$normal, normal1 = ps2$normal1, cancer = ps2$cancer, metastasis = ps2$metastasis) dat2 = suppressMessages(melt(dat2)) colnames(dat2) <- c("Breast.tissue", "HD") idx <- which(is.na(dat2$HD))
dat2 <- dat2[-idx, ] # To remove missing data

p1 = ggplot(dat1, aes(x = HD, fill = Breast.tissue, colour = Breast.tissue)) +
geom_histogram(alpha = 0.5, bins = 50, position = "identity",
na.rm = TRUE, size = 0.7) + ylab("Counts") + xlim(0,30) +
theme(axis.title.x = element_text(face = "bold", size = 20),
axis.text.x = element_text(face = "bold", size = 20, color = "black",
hjust = 0.5, vjust = 0.75),
axis.text.y = element_text(face = "bold", size = 20, color = "black"),
axis.title.y = element_text(face = "bold", size = 20,color = "black"),
legend.text = element_text(size = 20, face = "bold"),
legend.title = element_text(size = 20, face = "bold") ) +
ggtitle("Histogram for Potential methylation signal. GGamma3P")

p2 = ggplot(dat1,aes(x = Breast.tissue, y = HD , fill = Breast.tissue)) +
geom_boxplot(na.rm = TRUE) + ylim(0, 15) +
stat_summary(fun.data = fun_length, geom = "text", na.rm = TRUE,
position = position_dodge(width = 0.9), vjust = 1,
size = 6, fontface = "bold") +
geom_hline(yintercept = 6, linetype="dashed", color = "red") +
theme(axis.title.x = element_text(face = "bold", size = 20),
axis.text.x = element_text(face = "bold", size = 20, color = "black",
hjust = 0.5, vjust = 0.75),
axis.text.y = element_text(face = "bold", size = 20, color = "black"),
axis.title.y = element_text(face = "bold", size = 20,color = "black"),
legend.position = "none" ) +
ggtitle("Boxplot for Potential methylation signal. GGamma3P")

p3 = ggplot(dat2, aes(x = HD, fill = Breast.tissue, colour = Breast.tissue)) +
geom_histogram(alpha = 0.5, bins = 50, position = "identity",
na.rm = TRUE, size = 0.7) + ylab("Counts") + xlim(0,30) +
theme(axis.title.x = element_text(face = "bold", size = 20),
axis.text.x = element_text(face = "bold", size = 20, color = "black",
hjust = 0.5, vjust = 0.75),
axis.text.y = element_text(face = "bold", size = 20, color = "black"),
axis.title.y = element_text(face = "bold", size = 20,color = "black"),
legend.text = element_text(size = 20, face = "bold"),
legend.title = element_text(size = 20, face = "bold")) +
ggtitle("Histogram for Potential methylation signal. Gamma3P")

p4 = ggplot(dat2,aes(x = Breast.tissue, y = HD , fill = Breast.tissue)) +
geom_boxplot(na.rm = TRUE) + ylim(0, 15) +
stat_summary(fun.data = fun_length, geom = "text", na.rm = TRUE,
position = position_dodge(width = 0.9), vjust = 1,
size = 6, fontface = "bold") +
geom_hline(yintercept = 6, linetype="dashed", color = "red") +
theme(axis.title.x = element_text(face = "bold", size = 20),
axis.text.x = element_text(face = "bold", size = 20, color = "black",
hjust = 0.5, vjust = 0.75),
axis.text.y = element_text(face = "bold", size = 20, color = "black"),
axis.title.y = element_text(face = "bold", size = 20,color = "black"),
legend.position = "none" ) +
ggtitle("Boxplot for Potential methylation signal. Gamma3P")

grid.arrange(p1, p2, p3, p4, ncol = 2)

The graphics show that “Gamma3P” provides more information on the discrimination of treatment potential DMPs from the control.

## Cutpoint estimation Laws of statistical physics can account for background

methylation, a response to thermal fluctuations that presumably functions in DNA stability (Sanchez and Mackenzie 2016; Sanchez et al. 2019). True signal is detected based on the optimal cutpoint (López-Ratón et al. 2014), which can be estimated from the area under the curve (AUC) of a receiver operating characteristic (ROC) curve built from a logistic regression performed with the potential signals from controls and treatments. The ROC AUC is equivalent to the probability that a randomly- chosen positive instance is ranked more highly than a randomly-chosen negative instance (Fawcett 2005). In the current context, the AUC is equivalent to the probability to distinguish a randomly-chosen methylation regulatory signal induced by the treatment from a randomly-chosen signal in the control.

The need for the application of (what is now known as) signal detection in cancer research was pointed out by Youden in the midst of the last century (Youden 1950). In the next example, the simple cutpoint estimation available in Methyl-IT is based on the application of Youden index (Youden 1950). Although cutpoints are estimated for a single variable, the classification performance can be evaluated for several variables and applying different model classifiers. A optimal cutpoint distinguishes disease stages from healthy individual. The performance of this classification is given in the output of function estimateCutPoint.

A model classifier can be requested for further predictions and its classification performance is also provided. Below, the selected model classifier is a quadratic discriminant analysis (QDA) (classifier1 = “qda”, clas.perf = TRUE). Four predictor variables are available: the Hellinger divergence of methylation levels (hdiv), total variation distance (TV, absolute difference of methylation levels), relative position of cytosine site in the chromosome (pos), and the logarithm base two of the probability to observe a Hellinger divergence value $$H$$ greater than the critical value $$H_{\alpha = 0.05}$$ (values given as probabilities in object PS, wprob).

Notice that the cutpoint can be estimated for any of the two currently available information divergences: Hellinger divergence (div.col = 9) or the total variation distance (with Bayesian correction, div.col = 8).

### Cutpoint based on Youden index

Null distribution “GGamma3P”:

cutpoint1 = estimateCutPoint(LR = PS1,
control.names = c("Breast_normal","Breast_normal1"),
treatment.names = c("Breast_cancer", "Breast_metastasis"),
simple = TRUE,
classifier1 = "pca.logistic",
column = c(hdiv = TRUE, bay.TV = TRUE,
wprob = TRUE, pos = TRUE),
n.pc = 4 , center = TRUE, scale = TRUE,
div.col = 9, clas.perf = TRUE)
cutpoint1$cutpoint ## [1] 4.383196 cutpoint1$testSetPerformance
## Confusion Matrix and Statistics
##
##           Reference
## Prediction   CT   TT
##         CT 1123 1076
##         TT 4031 9874
##
##                Accuracy : 0.6829
##                  95% CI : (0.6756, 0.6901)
##     No Information Rate : 0.68
##     P-Value [Acc > NIR] : 0.2162
##
##                   Kappa : 0.141
##
##  Mcnemar's Test P-Value : <2e-16
##
##             Sensitivity : 0.9017
##             Specificity : 0.2179
##          Pos Pred Value : 0.7101
##          Neg Pred Value : 0.5107
##              Prevalence : 0.6800
##          Detection Rate : 0.6131
##    Detection Prevalence : 0.8635
##       Balanced Accuracy : 0.5598
##
##        'Positive' Class : TT
## 
cutpoint1$testSetModel.FDR ## [1] 0.2898957 Null distribution “Gamma3P”: cutpoint2 = estimateCutPoint(LR = PS2, control.names = c("Breast_normal","Breast_normal1"), treatment.names = c("Breast_cancer", "Breast_metastasis"), simple = TRUE, classifier1 = "pca.logistic", column = c(hdiv = TRUE, bay.TV = TRUE, wprob = TRUE, pos = TRUE), n.pc = 4 , center = TRUE, scale = TRUE, div.col = 9, clas.perf = TRUE) cutpoint2$cutpoint
## [1] 6.189402
cutpoint2$testSetPerformance ## Confusion Matrix and Statistics ## ## Reference ## Prediction CT TT ## CT 657 0 ## TT 0 1551 ## ## Accuracy : 1 ## 95% CI : (0.9983, 1) ## No Information Rate : 0.7024 ## P-Value [Acc > NIR] : < 2.2e-16 ## ## Kappa : 1 ## ## Mcnemar's Test P-Value : NA ## ## Sensitivity : 1.0000 ## Specificity : 1.0000 ## Pos Pred Value : 1.0000 ## Neg Pred Value : 1.0000 ## Prevalence : 0.7024 ## Detection Rate : 0.7024 ## Detection Prevalence : 0.7024 ## Balanced Accuracy : 1.0000 ## ## 'Positive' Class : TT ##  cutpoint2$testSetModel.FDR
## [1] 0

The cutpoint estimated based “GGamma3P” model is lower than the value based on “Gamma3P” model. Often, this is a good detail, since it permits the identification of more sites with high probability to carry methylation signal. However, the evaluation of the classification performance has the last word on model selection. In the current case, the DMP classification performance is much better for “Gamma3P” model than for “GGamma3P”.

In particular, the probabilities derived from each distribution model provide fundamental information to the model classifier. Hence, the probability that a potential DMP could carry a methylation signal estimated using “Gamma3P” model is more informative (for the model classifier) than the probability estimated using “GGamma3P” model.

### Cutpoint estimated using a model classifier

In the current case, the DMP classification performance for the signal derived with null distribution “Gamma3P” can be improved searching for an optimal cutpoint with a model classifier and using the total variation distance ( with Bayesian correction, bay.TV).

cutpoint2 = estimateCutPoint(LR = PS2,
control.names = c("Breast_normal","Breast_normal1"),
treatment.names = c("Breast_cancer", "Breast_metastasis"),
simple = FALSE,
classifier1 = "pca.logistic",
classifier2 = "pca.qda",
column = c(hdiv = TRUE, bay.TV = TRUE,
wprob = TRUE, pos = TRUE),
n.pc = 4 , center = TRUE, scale = TRUE,
div.col = 8)
cutpoint2$cutpoint ## [1] 0.5000034 cutpoint2$testSetPerformance
## Confusion Matrix and Statistics
##
##           Reference
## Prediction   CT   TT
##         CT 5220    0
##         TT    0 1550
##
##                Accuracy : 1
##                  95% CI : (0.9995, 1)
##     No Information Rate : 0.771
##     P-Value [Acc > NIR] : < 2.2e-16
##
##                   Kappa : 1
##
##  Mcnemar's Test P-Value : NA
##
##             Sensitivity : 1.000
##             Specificity : 1.000
##          Pos Pred Value : 1.000
##          Neg Pred Value : 1.000
##              Prevalence : 0.229
##          Detection Rate : 0.229
##    Detection Prevalence : 0.229
##       Balanced Accuracy : 1.000
##
##        'Positive' Class : TT
## 
cutpoint2$testSetModel.FDR ## [1] 0 The good DMP classification performance was kept at lower cutpoint value. ## DMPs Cytosine sites carrying a methylation signal are designated differentially informative methylated positions (DMPs). The probability that a DMP is not induced by the treatment is given by the probability of false alarm (P_FA, false positive, Sanchez et al. 2019). That is, the biological signal is naturally present in the control as well as in the treatment. According to the classical view in the epigenomic field, a DMP is a cytosine position carrying a significant methylation signal, which might be identified as differentially methylated cytosine site by Fisher’s exact test (or other current tests). In Methyl-IT, a DMP is a DNA cytosine position with high probability to be differentially hyper-methylated or hypo-methylated in the treatment with respect to a given control (reference individual). Notice that Methyl- IT definition of DMP is not deterministic in an ordinary sense, but stochastic- deterministic in physico-mathematical terms. In other words, if a given cytosine position is identified as a hyper-methylated DMP in the treatment, then with high probability such a site will be found more frequently hyper- methylated than hypo-methylated in the treatment population*_. Observe that it does not mean that, for all arbitrary subset of samples, the assumed cytosine position will be hypermethylated. DMPs are selected with the function selectDIMP DMPs = selectDIMP(PS2, div.col = 8, cutpoint = cutpoint2$cutpoint)

### The descriptive statistic of the read counts at DMPs

After the signal detection step the read counts at DMPs is notable greater than at the general methylated cytosine background:

# Maximum coverage at each cytosine site
methc <- lapply(DMPs, function(x) {
r <- apply(cbind(x$c1, x$c2), 1, max)
return(r)
})

# Quantiles
do.call(rbind, lapply(methc, function(x) {
q10 <- quantile(x, 0.1)
q60 <- quantile(x, 0.6)
q9999 <- quantile(x, 0.9999)
idx1 <- which(x >= q60)
idx2 <- which(x <= 500)
q95 <- quantile(x, 0.95)

idx <- intersect(idx1, idx2)
return(c(round(summary(x)), q10, q60,
quantile(x, c(0.95, 0.99, 0.999, 0.9999)),
num.siteGreater_8 = sum(x >= 8),
q60_to_500 = sum((x >= q60) & (x <= 500)),
num.siteGreater_500 = sum(x > 500)))})
)
##                   Min. 1st Qu. Median Mean 3rd Qu. Max.  10% 60% 95%   99%
## Breast_normal       18      32     42   41      48   91 27.0  44  62  70.0
## Breast_cancer       19      41     57   61      80  228 33.3  64 106 123.0
## Breast_metastasis   17      38     47   50      61  146 30.0  52  83 101.5
## Breast_normal1      23      29     34   37      45   76 26.0  38  53  60.0
##                     99.9%   99.99% num.siteGreater_8 q60_to_500
## Breast_normal      83.000  90.1082              8919       3872
## Breast_cancer     157.677 212.2036              2324        942
## Breast_metastasis 116.800 141.8150              1551        643
## Breast_normal1     71.000  74.3480              4131       1656
##                   num.siteGreater_500
## Breast_normal                       0
## Breast_cancer                       0
## Breast_metastasis                   0
## Breast_normal1                      0

The minimum amount of methylated read counts is observed “Breast_metastasis” (15 reads) and for any the samples the tenth percent of DMPs is 26 or more reads.

### Venn Diagram of DMPs

The Venn diagram of DMPs reveals that the number cytosine site carrying methylation signal with a divergence level comparable to that observed in breast tissues with cancer and metastasis is relatively small (2797 DMPs). The number of DMPs decreased in the breast tissue with metastasis, but, as shown in the last boxplot, the intensity of the signal increased.

n12 = length(GenomicRanges::intersect(DMPs$Breast_normal, DMPs$Breast_cancer))
n13 = length(GenomicRanges::intersect(DMPs$Breast_normal, DMPs$Breast_metastasis))
n23 = length(GenomicRanges::intersect(DMPs$Breast_cancer, DMPs$Breast_metastasis))
n123 = length(Reduce(GenomicRanges::intersect,
list(DMPs$Breast_normal, DMPs$Breast_cancer,
DMPs$Breast_metastasis))) grid.newpage() v = draw.triple.venn(area1 = length(DMPs$Breast_normal),
area2 = length(DMPs$Breast_cancer), area3 = length(DMPs$Breast_metastasis),
n12 = n12, n23 = n23, n13 = n13, n123 = n123,
category = c("Breast_normal", "Breast_cancer",
"Breast_metastasis"),
lty = rep("blank", 3), fill = c("blue", "yellow",
"magenta"),
alpha = c(0.1, 0.2, 0.3),
cat.pos = c(-80, 90, 0),
cat.col = c("blue", "darkorange", "red"),
cat.dist = c( -0.1, -0.08, -0.26),
cex = rep(1.7, 7),
cat.cex = c( 1.5, 1.5, 1.5),
label.col = c( "blue", "darkorange", "darkorange",
"red", "white", "red", "red"),
scaled = TRUE)
grid.draw(v)

Notice that natural methylation regulatory signals (not induced by the treatment) are present in both groups, control and treatment. The signal detection step permits us to discriminate the “ordinary” signals observed in the control from those induced by the treatment (a disease in the current case). In addition, this diagram reflects a classification of DMPs only based on the cytosine positions. That is, this Venn diagram cannot tell us whether DMPs at the same position can be distinguishable or not. For example, DMPs at the same positions in control and treatment can happen with different probabilities estimated from their corresponding fitted Weibull distributions (see below).

### Monte Carlo Evaluation of the Model Classification

Performance The regulatory methylation signal is an output from a natural process that continuously takes place across the ontogenetic development of the organism. Therefore, we expect to see methylation signal in natural, ordinary conditions. Function ‘evaluateDIMPclass’ can be used to perform a classification of DMPs into two classes: DMPS from control and DMPs from treatment samples, as well as an evaluation of the classification performance (for more details see evaluateDIMPclass).

As a matter of fact this function is called by function estimateCutPoint. Additional feature not used before is the possibility to perform Monte Carlo evaluation of the model classifier performance. In above split of the sample into two subsets, training and test datasets, we would be just lucky getting the right proportion that yields the best classification performance. What about is the random split is repeat 300 times?

The performance of the “pca.qda” classifier is:

performance <- evaluateDIMPclass(LR = DMPs,
control.names = c("Breast_normal", "Breast_normal1"),
treatment.names = c("Breast_cancer", "Breast_metastasis"),
column = c(hdiv = TRUE, bay.TV = TRUE,
wprob = TRUE, pos = TRUE),
classifier = "pca.qda", prop = 0.6,
n.pc = 4 , center = TRUE, scale = TRUE,
num.boot = 300, output = "mc.val",
num.cores = 6L, tasks = 2L)
performance
##     Accuracy     Kappa   AccuracyLower    AccuracyUpper  AccuracyNull
##  Min.   :1   Min.   :1   Min.   :0.9995   Min.   :1     Min.   :0.771
##  1st Qu.:1   1st Qu.:1   1st Qu.:0.9995   1st Qu.:1     1st Qu.:0.771
##  Median :1   Median :1   Median :0.9995   Median :1     Median :0.771
##  Mean   :1   Mean   :1   Mean   :0.9995   Mean   :1     Mean   :0.771
##  3rd Qu.:1   3rd Qu.:1   3rd Qu.:0.9995   3rd Qu.:1     3rd Qu.:0.771
##  Max.   :1   Max.   :1   Max.   :0.9995   Max.   :1     Max.   :0.771
##
##  AccuracyPValue McnemarPValue  Sensitivity  Specificity Pos Pred Value
##  Min.   :0      Min.   : NA   Min.   :1    Min.   :1    Min.   :1
##  1st Qu.:0      1st Qu.: NA   1st Qu.:1    1st Qu.:1    1st Qu.:1
##  Median :0      Median : NA   Median :1    Median :1    Median :1
##  Mean   :0      Mean   :NaN   Mean   :1    Mean   :1    Mean   :1
##  3rd Qu.:0      3rd Qu.: NA   3rd Qu.:1    3rd Qu.:1    3rd Qu.:1
##  Max.   :0      Max.   : NA   Max.   :1    Max.   :1    Max.   :1
##                 NA's   :300
##  Neg Pred Value   Precision     Recall        F1      Prevalence
##  Min.   :1      Min.   :1   Min.   :1   Min.   :1   Min.   :0.229
##  1st Qu.:1      1st Qu.:1   1st Qu.:1   1st Qu.:1   1st Qu.:0.229
##  Median :1      Median :1   Median :1   Median :1   Median :0.229
##  Mean   :1      Mean   :1   Mean   :1   Mean   :1   Mean   :0.229
##  3rd Qu.:1      3rd Qu.:1   3rd Qu.:1   3rd Qu.:1   3rd Qu.:0.229
##  Max.   :1      Max.   :1   Max.   :1   Max.   :1   Max.   :0.229
##
##  Detection Rate  Detection Prevalence Balanced Accuracy
##  Min.   :0.229   Min.   :0.229        Min.   :1
##  1st Qu.:0.229   1st Qu.:0.229        1st Qu.:1
##  Median :0.229   Median :0.229        Median :1
##  Mean   :0.229   Mean   :0.229        Mean   :1
##  3rd Qu.:0.229   3rd Qu.:0.229        3rd Qu.:1
##  Max.   :0.229   Max.   :0.229        Max.   :1
## 

## Differentially methylated genomic regions (DMRs)

Our degree of confidence in whether DMP counts in both groups of samples, control and treatment, represent true biological signal was determined in the signal detection step. To estimate DIMRs, we followed similar steps to those proposed in Bioconductor R package DESeq2 (Love, Huber, and Anders 2014), but our GLM test looks for statistical difference between the groups based on gene-body DMP counts overlapping a given genomic region rather than read counts. The regression analysis of the generalized linear model (GLM) with logarithmic link was applied to test the difference between group counts.

The fitting algorithmic approaches provided by ‘glm’ and ‘glm.nb’ functions from the R packages stat and MASS, respectively, were used for Poisson (PR), Quasi-Poisson (QPR) and Negative Binomial (NBR) linear regression analyses, respectively.

In Methyl-IT, the concept of DMR is generalized and it is not limited to any particular genomic region found with specific clustering algorithm. It can be applied to any naturally or algorithmically defined genomic region. For example, an exon region identified statistically to be differentially methylated by using generalized linear regression model (GLM) is a DMR. Differentially methylated genes (DMGs) are estimated from group comparisons for the number of DMPs on gene-body regions between control and treatment. This permits the extension of the concept of DMR by considering a differentially methylated gene as a particular case of DMRs.

### Differentially methylated genes (DMGs)

We shall call DMGs those DIMRs restricted to gene-body regions. DMGs are detected using function countTest2. Gene annotation is taken from Ensembl

# To load human gene annotation
# AG = import(con = paste0("ftp://ftp.ensembl.org/pub/release-91/gff3/",
# "homo_sapiens/Homo_sapiens.GRCh38.91.gff3.gz"))
# genes = AG[ AG$type == "gene", c( "gene_id", "biotype", "Name" ) ] # genes = genes[ genes$biotype == "protein_coding", "gene_id" ]
# seqlevels(genes, "coarse") 

Here, for the sake of brevity, we just load the “RData” compressed file containing only the gene annotation:

con = url(paste0("https://git.psu.edu/genomath/MethylIT_examples/raw/master/",
"Homo_sapiens_GRCh38.91_genes.RData"))
genes = genes[ genes$biotype == "protein_coding", ] seqlevels(genes, "coarse") <- "13" # To keep a consistent chromosome annotation seqlevels(genes) <- "chr13" genes ## GRanges object with 323 ranges and 3 metadata columns: ## seqnames ranges strand | gene_id biotype ## <Rle> <IRanges> <Rle> | <character> <character> ## [1] chr13 19173770-19181852 - | ENSG00000198033 protein_coding ## [2] chr13 19422877-19536762 - | ENSG00000132958 protein_coding ## [3] chr13 19633681-19673459 + | ENSG00000196199 protein_coding ## [4] chr13 19674752-19783019 - | ENSG00000121390 protein_coding ## [5] chr13 19823482-19863636 - | ENSG00000132950 protein_coding ## ... ... ... ... . ... ... ## [319] chr13 113977783-114132611 - | ENSG00000185989 protein_coding ## [320] chr13 114179238-114223084 + | ENSG00000283361 protein_coding ## [321] chr13 114234887-114272723 + | ENSG00000130177 protein_coding ## [322] chr13 114281584-114305817 + | ENSG00000169062 protein_coding ## [323] chr13 114314513-114327328 + | ENSG00000198824 protein_coding ## Name ## <character> ## [1] TUBA3C ## [2] TPTE2 ## [3] MPHOSPH8 ## [4] PSPC1 ## [5] ZMYM5 ## ... ... ## [319] RASA3 ## [320] AL160396.2 ## [321] CDC16 ## [322] UPF3A ## [323] CHAMP1 ## ------- ## seqinfo: 1 sequence from an unspecified genome; no seqlengths Function getDIMPatGenes is used to count the number of DMPs at gene-body. Nevertheless, it can be used for any arbitrary specified genomic region as well. The operation of this function is based on the ‘findOverlaps’ function from the ‘GenomicRanges’ Bioconductor R package. The ‘findOverlaps’ function has several critical parameters like, for example, ‘maxgap’, ‘minoverlap’, and ‘ignore.strand’. In our function getDIMPatGenes, except for setting ignore.strand = TRUE and type = “within”, we preserve the rest of default ‘findOverlaps’ parameters. In this case, these are important parameter settings because the local mechanical effect of methylation changes on a DNA region where a gene is located is independent of the strand where the gene is encoded. That is, methylation changes located in any of the two DNA strands inside the gene-body region will affect the flexibility of the DNA molecule (Choy et al. 2010; Severin et al. 2011). DMPsBN = getDIMPatGenes(GR = DMPs$Breast_normal, GENES = genes)
DMPsBN1 = getDIMPatGenes(GR = DMPs$Breast_normal1, GENES = genes) DMPsBC = getDIMPatGenes(GR = DMPs$Breast_cancer, GENES = genes)
DMPsBM = getDIMPatGenes(GR = DMPs$Breast_metastasis, GENES = genes) The number of DMPs on the strand where a gene is encoded is obtained by setting ignore.strand = FALSE. However, for the current example results will be the same since the datasets downloaded from GEO do not have strand information. Next, the above GRanges objects carrying the DMP counts from each sample are grouped into a single GRanges object. Since we have only one control, to perform group comparison and to move forward with this example, we duplicated ‘Breast_normal’ sample. Obviously, the confidence on the results increases with the number of sample replications per group (in this case, it is only an illustrative example on how to perform the analysis, since a fair comparison requires for more than one replicate in the control group). Genes.DMPs = uniqueGRanges( list(DMPsBN[, 2], DMPsBN1[, 2], DMPsBC[, 2], DMPsBM[, 2]), type = "equal", verbose = FALSE, ignore.strand = TRUE ) colnames( mcols(Genes.DMPs)) <- c("Breast_normal", "Breast_normal1", "Breast_cancer", "Breast_metastasis") Next, the set of mapped genes are annotated GeneID = subsetByOverlaps(genes, Genes.DMPs, type = "equal", ignore.strand = FALSE) names( Genes.DMPs ) <- GeneID$gene_id
Genes.DMPs
## GRanges object with 270 ranges and 4 metadata columns:
##                   seqnames              ranges strand | Breast_normal
##                      <Rle>           <IRanges>  <Rle> |     <numeric>
##   ENSG00000198033    chr13   19173770-19181852      * |             1
##   ENSG00000132958    chr13   19422877-19536762      * |             1
##   ENSG00000196199    chr13   19633681-19673459      * |             2
##   ENSG00000121390    chr13   19674752-19783019      * |             5
##   ENSG00000132950    chr13   19823482-19863636      * |             1
##               ...      ...                 ...    ... .           ...
##   ENSG00000185989    chr13 113977783-114132611      * |            65
##   ENSG00000283361    chr13 114179238-114223084      * |            27
##   ENSG00000130177    chr13 114234887-114272723      * |            22
##   ENSG00000169062    chr13 114281584-114305817      * |            16
##   ENSG00000198824    chr13 114314513-114327328      * |             0
##                   Breast_normal1 Breast_cancer Breast_metastasis
##                        <numeric>     <numeric>         <numeric>
##   ENSG00000198033              0             0                 0
##   ENSG00000132958              2             0                 0
##   ENSG00000196199              1             0                 0
##   ENSG00000121390              2             0                 3
##   ENSG00000132950              0             0                 0
##               ...            ...           ...               ...
##   ENSG00000185989             34            45                14
##   ENSG00000283361              7            21                14
##   ENSG00000130177              7            13                 6
##   ENSG00000169062              7            10                 5
##   ENSG00000198824              0             0                 1
##   -------
##   seqinfo: 1 sequence from an unspecified genome; no seqlengths

Now, we build a glmDataSet object.

## An experiment design is set.
colData <- data.frame(condition = factor(c("BN","BN","BC","BC"),
levels = c("BN", "BC")),
c("Breast_normal","Breast_normal1",
"Breast_cancer","Breast_metastasis"),
row.names = 2)
## A RangedGlmDataSet is created
ds <- glmDataSet(GR = Genes.DMPs, colData = colData)

DMG analysis is performed with the function countTest2

DMGs = countTest2(DS = ds, num.cores = 4L, minCountPerIndv = 8,
countFilter = TRUE, maxGrpCV = c(1, 1),
Minlog2FC = 1, pvalCutOff = 0.05,
MVrate = .95, verbose = FALSE)
DMGs
## GRanges object with 64 ranges and 12 metadata columns:
##                   seqnames              ranges strand | Breast_normal
##                      <Rle>           <IRanges>  <Rle> |     <numeric>
##   ENSG00000165475    chr13   20403667-20525857      * |            41
##   ENSG00000150457    chr13   20973032-21061547      * |            29
##   ENSG00000165487    chr13   21492691-21604214      * |            31
##   ENSG00000102699    chr13   24420926-24512810      * |            15
##   ENSG00000139505    chr13   25246201-25288009      * |            10
##               ...      ...                 ...    ... .           ...
##   ENSG00000204442    chr13 107163510-107866735      * |            84
##   ENSG00000041515    chr13 108596152-109208007      * |            16
##   ENSG00000126218    chr13 113122814-113149529      * |            15
##   ENSG00000185896    chr13 113297241-113323672      * |            15
##   ENSG00000184497    chr13 113759240-113816995      * |            15
##                   Breast_normal1 Breast_cancer Breast_metastasis
##                        <numeric>     <numeric>         <numeric>
##   ENSG00000165475             19             9                 7
##   ENSG00000150457             16             6                 9
##   ENSG00000165487             10             4                10
##   ENSG00000102699              6             1                 2
##   ENSG00000139505              6             0                 0
##               ...            ...           ...               ...
##   ENSG00000204442             44             8                 3
##   ENSG00000041515              7             3                 1
##   ENSG00000126218              9             2                 5
##   ENSG00000185896              5             1                 2
##   ENSG00000184497              6             4                 3
##                              log2FC  scaled.deviance               pvalue
##                           <numeric>        <numeric>            <numeric>
##   ENSG00000165475 -1.32175583998232 4.96208101271995    0.030606129674274
##   ENSG00000150457 -1.09861228866811 4.65828980395393   0.0361102854476026
##   ENSG00000165487 -1.07451473708905 4.24757904423761   0.0397695740962727
##   ENSG00000102699 -1.94591014905531 7.54897971118099  0.00889363821746184
##   ENSG00000139505 -2.19722457733622 10.1593402974678  0.00558981768966344
##               ...               ...              ...                  ...
##   ENSG00000204442 -2.45413499112125 38.0399528175922 6.01036075344073e-09
##   ENSG00000041515 -1.74919985480926 5.45838587602237   0.0232156201855052
##   ENSG00000126218 -1.23214368129263 4.09418109306589   0.0481345693904388
##   ENSG00000185896 -1.89711998488588 6.35686440652162   0.0162045338242277
##   ENSG00000184497 -1.09861228866811 5.78113844812565   0.0222108197326888
##                            model             adj.pval     CT.SignalDensity
##                         <factor>            <numeric>            <numeric>
##   ENSG00000165475 Neg.Binomial.W   0.0362739314658062  0.00024551726395561
##   ENSG00000150457 Neg.Binomial.W   0.0420192412481194 0.000254191332640427
##   ENSG00000165487   Neg.Binomial   0.0442973653695546  0.00018381693626484
##   ENSG00000102699 Neg.Binomial.W   0.0167409660563987 0.000114273276378081
##   ENSG00000139505 Neg.Binomial.W   0.0143099332855384   0.0001913463608314
##               ...            ...                  ...                  ...
##   ENSG00000204442   Neg.Binomial 3.84663088220207e-07 9.10091492635369e-05
##   ENSG00000041515 Neg.Binomial.W   0.0309541602473402  1.8795272090165e-05
##   ENSG00000126218 Neg.Binomial.W   0.0487745441069833  0.00044916903728103
##   ENSG00000185896 Neg.Binomial.W   0.0246926229702518 0.000378329297820823
##   ENSG00000184497   Neg.Binomial   0.0302445204870656 0.000181799293579888
##                       TT.SignalDensity SignalDensityVariation
##                              <numeric>              <numeric>
##   ENSG00000165475 6.54712703881628e-05  -0.000180045993567448
##   ENSG00000150457 8.47304442134755e-05  -0.000169460888426951
##   ENSG00000165487 6.27667587245795e-05   -0.00012105017754026
##   ENSG00000102699 1.63247537682973e-05   -9.7948522609784e-05
##   ENSG00000139505                    0    -0.0001913463608314
##               ...                  ...                    ...
##   ENSG00000204442  7.8210987648352e-06  -8.31880504987017e-05
##   ENSG00000041515 3.26874297220261e-06  -1.55265291179624e-05
##   ENSG00000126218 0.000131007635873634  -0.000318161401407396
##   ENSG00000185896 5.67493946731235e-05    -0.0003215799031477
##   ENSG00000184497 6.05997645266293e-05  -0.000121199529053259
##   -------
##   seqinfo: 1 sequence from an unspecified genome; no seqlengths

Here, we adds the gene alias:

hits = findOverlaps(DMGs, genes, type = "equal")
DMGs = DMGs[queryHits(hits)]
DMGs$alias <- genes$Name[subjectHits(hits)]

BRCA2, a breast cancer associated risk gene, is found between the DMGs

# DMGs
DMGs[ grep( "ENSG00000139618", names(DMGs) ) ]
## GRanges object with 1 range and 13 metadata columns:
##                   seqnames            ranges strand | Breast_normal
##                      <Rle>         <IRanges>  <Rle> |     <numeric>
##   ENSG00000139618    chr13 32315474-32400266      * |            22
##                   Breast_normal1 Breast_cancer Breast_metastasis
##                        <numeric>     <numeric>         <numeric>
##   ENSG00000139618             11             0                 0
##                              log2FC  scaled.deviance               pvalue
##                           <numeric>        <numeric>            <numeric>
##   ENSG00000139618 -2.86220088092947 18.3766614005661 0.000363516387975643
##                            model            adj.pval    CT.SignalDensity
##                         <factor>           <numeric>           <numeric>
##   ENSG00000139618 Neg.Binomial.W 0.00290813110380514 0.00019459153467857
##                   TT.SignalDensity SignalDensityVariation       alias
##                          <numeric>              <numeric> <character>
##   ENSG00000139618                0   -0.00019459153467857       BRCA2
##   -------
##   seqinfo: 1 sequence from an unspecified genome; no seqlengths

## Concluding remarks

Results fundamentally depend on the use of the proper reference samples. If there is not enough control samples to create an independent reference virtual individual, then the centroid from the control group can be used instead, which can be built with function poolFromGRlist.

The reference is used to put individuals from control and patients in the same system of reference. The information divergences of methylation levels with respect to the reference individual are estimated for each cytosine site for each individual, from control and treatment. As a result, the information on the natural spontaneous variation in the control and treatment populations is carried in the probability distributions of the information divergences, estimated for each individual. It does not matter how statistically significant a DMP would be in the group of patients.

What really matter is how big is the probability to observe the same methylation event in the control group in respect to the patient group. Decisions are ultimately made on the basis of such classification probabilities at single cytosine positions. The signal detection based approach implemented in Methyl-IT is addressed to confront the mentioned issue and, in consequence, the level of resolution of Methyl-IT approach reach the single cytosine site.

## Acknowledgments

We thank Professor David J Miller for valuable conversations and suggestions on our mathematical modeling.

## Funding The work was supported by funding from NSF-SBIR

(2015-33610-23428-UNL) and the Bill and Melinda Gates Foundation (OPP1088661).

## S2. Session Information

## R version 3.6.3 (2020-02-29)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 18.04.4 LTS
##
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1
##
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## attached base packages:
##  [1] grid      parallel  stats4    stats     graphics  grDevices utils
##  [8] datasets  methods   base
##
## other attached packages:
##  [1] VennDiagram_1.6.20   futile.logger_1.4.3  gridExtra_2.3
##  [4] reshape2_1.4.3       ggplot2_3.3.0        MethylIT_0.3.2
##  [7] rtracklayer_1.44.4   GenomicRanges_1.36.1 GenomeInfoDb_1.20.0
## [10] IRanges_2.18.3       S4Vectors_0.22.1     BiocGenerics_0.30.0
## [13] knitr_1.28
##
## loaded via a namespace (and not attached):
##  [1] colorspace_1.4-1            ellipsis_0.3.0
##  [3] class_7.3-16                rprojroot_1.3-2
##  [5] XVector_0.24.0              ArgumentCheck_0.10.2
##  [7] fs_1.4.0                    rstudioapi_0.11
##  [9] farver_2.0.3                bit64_0.9-7
## [11] AnnotationDbi_1.46.1        prodlim_2019.11.13
## [13] fansi_0.4.1                 lubridate_1.7.4
## [15] codetools_0.2-16            splines_3.6.3
## [17] pROC_1.16.2                 Rsamtools_2.0.3
## [19] caret_6.0-86                annotate_1.56.1
## [21] compiler_3.6.3              backports_1.1.5
## [23] assertthat_0.2.1            Matrix_1.2-18
## [25] cli_2.0.2                   formatR_1.7
## [27] htmltools_0.4.0             tools_3.6.3
## [29] gtable_0.3.0                glue_1.3.2
## [31] GenomeInfoDbData_1.2.1      dplyr_0.8.5
## [33] Rcpp_1.0.4                  Biobase_2.44.0
## [35] pkgdown_1.5.0               vctrs_0.2.4
## [37] Biostrings_2.52.0           nlme_3.1-145
## [39] iterators_1.0.12            timeDate_3043.102
## [41] gower_0.2.1                 xfun_0.12
## [43] stringr_1.4.0               lifecycle_0.2.0
## [45] XML_3.99-0.3                zlibbioc_1.30.0
## [47] MASS_7.3-51.5               scales_1.1.0
## [49] ipred_0.9-9                 SummarizedExperiment_1.14.1
## [51] lambda.r_1.2.4              yaml_2.2.1
## [53] memoise_1.1.0               rpart_4.1-15
## [55] stringi_1.4.6               RSQLite_2.2.0
## [57] genefilter_1.66.0           desc_1.2.0
## [59] foreach_1.5.0               e1071_1.7-3
## [61] BiocParallel_1.18.1         lava_1.6.7
## [63] rlang_0.4.5                 pkgconfig_2.0.3
## [65] matrixStats_0.56.0          bitops_1.0-6
## [67] evaluate_0.14               lattice_0.20-40
## [69] purrr_0.3.3                 GenomicAlignments_1.20.1
## [71] recipes_0.1.10              labeling_0.3
## [73] bit_1.1-15.2                tidyselect_1.0.0
## [75] plyr_1.8.6                  magrittr_1.5
## [77] nls2_0.2                    R6_2.4.1
## [79] generics_0.0.2              DelayedArray_0.10.0
## [81] DBI_1.1.0                   pillar_1.4.3
## [83] withr_2.1.2                 survival_3.1-11
## [85] RCurl_1.98-1.1              nnet_7.3-13
## [87] tibble_3.0.0                crayon_1.3.4
## [89] futile.options_1.0.1        rmarkdown_2.1.1
## [91] minpack.lm_1.2-1            data.table_1.12.8
## [93] blob_1.2.1                  ModelMetrics_1.2.2.2
## [95] digest_0.6.25               xtable_1.8-4
## [97] munsell_0.5.0

## References.

Baldi, Pierre, and Soren Brunak. 2001. Bioinformatics: the machine learning approach. Second. Cambridge: MIT Press.

Basu, A., A. Mandal, and L. Pardo. 2010. “Hypothesis testing for two discrete populations based on the Hellinger distance.” Statistics & Probability Letters 80 (3-4). Elsevier B.V.: 206–14. https://doi.org/10.1016/j.spl.2009.10.008.

Carter, Jane V., Jianmin Pan, Shesh N. Rai, and Susan Galandiuk. 2016. “ROC-ing along: Evaluation and interpretation of receiver operating characteristic curves.” Surgery 159 (6). Mosby: 1638–45. https://doi.org/10.1016/j.surg.2015.12.029.

Choy, John S., Sijie Wei, Ju Yeon Lee, Song Tan, Steven Chu, and Tae Hee Lee. 2010. “DNA methylation increases nucleosome compaction and rigidity.” Journal of the American Chemical Society 132 (6). American Chemical Society: 1782–3. https://doi.org/10.1021/ja910264z.

Dolzhenko, Egor, and Andrew D Smith. 2014. “Using beta-binomial regression for high-precision differential methylation analysis in multifactor whole-genome bisulfite sequencing experiments.” BMC Bioinformatics 15 (1). BioMed Central: 215. https://doi.org/10.1186/1471-2105-15-215.

Fawcett, Tom. 2005. “An introduction to ROC analysis.” https://doi.org/10.1016/j.patrec.2005.10.010.

Greiner, M, D Pfeiffer, and R D Smith. 2000. “Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests.” Preventive Veterinary Medicine 45 (1-2): 23–41. https://doi.org/10.1016/S0167-5877(00)00115-X.

Harpaz, Rave, William DuMouchel, Paea LePendu, Anna Bauer-Mehren, Patrick Ryan, and Nigam H Shah. 2013. “Performance of Pharmacovigilance Signal Detection Algorithms for the FDA Adverse Event Reporting System.” Clin Pharmacol Ther 93 (6): 1–19. https://doi.org/10.1038/clpt.2013.24.Performance.

Hebestreit, Katja, Martin Dugas, and Hans-Ulrich Klein. 2013. “Detection of significantly differentially methylated regions in targeted bisulfite sequencing data.” Bioinformatics (Oxford, England) 29 (13): 1647–53. https://doi.org/10.1093/bioinformatics/btt263.

Kruspe, Sven, David D. Dickey, Kevin T. Urak, Giselle N. Blanco, Matthew J. Miller, Karen C. Clark, Elliot Burghardt, et al. 2017. “Rapid and Sensitive Detection of Breast Cancer Cells in Patient Blood with Nuclease-Activated Probe Technology.” Molecular Therapy - Nucleic Acids 8 (September). Cell Press: 542–57. https://doi.org/10.1016/j.omtn.2017.08.004.

Love, M I, W Huber, and S Anders. 2014. “Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2.” Genome Biology 15 (12): 1–34. https://doi.org/Artn 550\rDoi 10.1186/S13059-014-0550-8.

López-Ratón, Mónica, Mar$$\backslash$$$$\backslash$$ia Xosé Rodríguez-Álvarez, Carmen Cadarso-Suárez, Francisco Gude-Sampedro, and Others. 2014. “OptimalCutpoints: an R package for selecting optimal cutpoints in diagnostic tests.” Journal of Statistical Software 61 (8). Foundation for Open Access Statistics: 1–36. https://www.jstatsoft.org/article/view/v061i08.

Ngo, Thuy T.M., Jejoong Yoo, Qing Dai, Qiucen Zhang, Chuan He, Aleksei Aksimentiev, and Taekjip Ha. 2016. “Effects of cytosine modifications on DNA flexibility and nucleosome mechanical stability.” Nature Communications 7 (February). Nature Publishing Group: 10813. https://doi.org/10.1038/ncomms10813.

Robinson, Mark D., Abdullah Kahraman, Charity W. Law, Helen Lindsay, Malgorzata Nowicka, Lukas M. Weber, and Xiaobei Zhou. 2014. “Statistical methods for detecting differentially methylated loci and regions.” Frontiers in Genetics 5 (SEP). Frontiers: 324. https://doi.org/10.3389/fgene.2014.00324.

Sanchez, Robersy, and Sally A. Mackenzie. 2016. “Information Thermodynamics of Cytosine DNA Methylation.” Edited by Barbara Bardoni. PLOS ONE 11 (3). Public Library of Science: e0150427. https://doi.org/10.1371/journal.pone.0150427.

Sanchez, Robersy, Xiaodong Yang, Thomas Maher, and Sally Mackenzie. 2019. “Discrimination of DNA Methylation Signal from Background Variation for Clinical Diagnostics.” Int. J. Mol. Sci. 20 (21): 5343. https://doi.org/https://doi.org/10.3390/ijms20215343.

Severin, Philip M D, Xueqing Zou, Hermann E Gaub, and Klaus Schulten. 2011. “Cytosine methylation alters DNA mechanical properties.” Nucleic Acids Research 39 (20): 8740–51. https://doi.org/10.1093/nar/gkr578.

Stevens, James P. 2009. Applied Multivariate Statistics for the Social Sciences. Fifth Edit. Routledge Academic.

Youden, W. J. 1950. “Index for rating diagnostic tests.” Cancer 3 (1): 32–35. https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3.