--- abstract: | In recent years, there has been increasing interest in developing foundation models for time series data that can generalize across diverse downstream tasks. While numerous forecasting-oriented foundation models have been introduced, there is a notable scarcity of models tailored for time series classification. To address this gap, we present **Mantis**, a new open-source foundation model for time series classification based on the Vision Transformer (ViT) architecture that has been pre-trained using a contrastive learning approach. Our experimental results show that Mantis outperforms existing foundation models both when the backbone is frozen and when fine-tuned, while achieving the lowest calibration error. In addition, we propose several adapters to handle the multivariate setting, reducing memory requirements and modeling channel interdependence. author: - '`\hspace{1cm}`{=latex}Vasilii Feofanov' - Songkang Wen - Marius Alonso - Romain Ilbert - Hongbo Guo - '`\vspace{-0.1cm}`{=latex} `\newline`{=latex}`\newline {Malik Tiomoko}`{=latex}' - Lujia Pan - Jianfeng Zhang - Ievgen Redko bibliography: - references.bib date: '`\vspace{-0.7cm}`{=latex}' title: '`\algoname`{=latex}: Lightweight Calibrated Foundation Model for User-Friendly Time Series Classification' --- ```{=latex} \newcommand*\pct{\mkern2mu\scalebox{.9}{\%}} ``` ```{=latex} \newcommand{\theHalgorithm}{\arabic{algorithm}} ``` ```{=latex} \newcommand{\algoname}{Mantis} ``` ```{=latex} \newcommand{\tick}{\pmb{\textcolor{ForestGreen}{$\bm{\checkmark}$}}~} ``` ```{=latex} \newcommand{\crossi}{\pmb{\textcolor{BrickRed}{$\times$}}~} ``` ```{=latex} \renewcommand\Authfont{\bfseries} ``` ```{=latex} \renewcommand\Authsep{,\quad} ``` ```{=latex} \renewcommand\Authand{ and } ``` ```{=latex} \renewcommand\Authands{,\quad} ``` ```{=latex} \renewcommand\theaffil{} ``` ```{=latex} \renewcommand\Affilfont{\normalfont} ``` ```{=latex} \newcommand{\firstauthor}[2]{\author[#1]{#2}} ``` ```{=latex} \newcommand{\myauthor}[2]{\author[#1]{#2}} ``` ```{=latex} \renewcommand{\shorttitle}{\algoname: Foundation Model for Time Series Classification} ``` ```{=latex} \newcommand{\mycite}[1]{\citeauthor{#1}, \citeyear{#1}} ``` ```{=latex} \maketitle ``` ```{=latex} \newcommand{\logo}{\raisebox{-0.2\height}{\includegraphics[height=0.5cm]{pics/mantis_logo.png}}} ``` ```{=latex} \centering ``` ```{=latex} \hspace{0.0cm} ``` `\scalebox{1.12}{\textbf{Mantis}}`{=latex}`\hspace{0.14cm}`{=latex}`\logo`{=latex}`\hspace{0.21cm}`{=latex} Introduction ============ The advent of large foundation models [@bommasani2021opportunities] in computer vision [@he2015resnet; @dosovitskiy2021vit] and natural language processing [@achiam2023gpt; @touvron2023llama] has significantly transformed research and applications. These models are pre-trained on extensive, diverse datasets to generalize across a wide range of downstream tasks. This approach not only simplifies model architecture selection but also reduces the need for large amounts of labeled data for new tasks, as the foundation model leverages its previously acquired knowledge. Over the past two years, the development of time series foundation models (TSFMs) has emerged as a prominent research area. For time series forecasting, numerous models have been proposed, either pre-trained from scratch on large volumes of time series data [@rasul2023lagllama; @das2023decoder; @woo2024moirai; @wang2024rose], including synthetic time series [@ansari2024chronos; @bhethanabhotla2024mamba4cast], or adapted from pre-trained large language models (LLMs, `\mycite{jin2023time}`{=latex}; `\mycite{chang2023llm4ts}`{=latex}; `\mycite{cao2023tempo}`{=latex}; `\mycite{xue2023promptcast}`{=latex}; `\mycite{gruver2024large}`{=latex}). Some foundation models are designed to handle multiple tasks simultaneously, such as forecasting, classification, imputation, and anomaly detection [@zhou2023onefitsall; @goswami2024moment]. However, these general-purpose models may exhibit suboptimal performance because certain pre-training schemes are inherently more suitable for specific tasks (e.g., masked reconstruction loss may be better suited for imputation, while contrastive loss aligns more naturally with classification). Surprisingly, despite the widespread use of time series classification [@bagnall2018uea; @dau2019ucr; @dempster2020rocket], relatively few models focus specifically on this task. To address this gap, we developed Mantis, a new time series classification foundation model: ```{=latex} \centering ``` *Our goal is to provide an open-source, lightweight, and high-performance model that can be readily used by practitioners and researchers across diverse domains.* The design of Mantis draws from advancements in time series representation learning [@yue2022ts2vec; @ijcai2021-324; @NEURIPS2022_194b8dac], transformer-based models for time series [@Yuqietal-2023-PatchTST; @ilbert2024samformer; @lin2024nutime]. The architecture of our model (Figure `\ref{fig:mantis-architecture}`{=latex}) features a token generator that transforms time series data into meaningful patches and a Vision Transformer (ViT) component similar to @dosovitskiy2021vit. We contrastively pre-trained Mantis on 7 million time series samples, resulting in a model with 8 million parameters. The pre-trained checkpoint is open-sourced on [HuggingFace](https://huggingface.co/paris-noah/Mantis-8M), and our [GitHub package](https://github.com/vfeofanov/mantis) includes an API for inference and fine-tuning on new downstream tasks. We perform extensive experiments to demonstrate the superiority of Mantis, while also showing that our model is the most calibrated compared to other time series classification foundation models. ```{=latex} \centering ``` ![Architecture. By symbol $+$ we denote the sum operator, while $||$ designates the vector concatenation operator.](pics/mantis_architecture.png){#fig:mantis-architecture width="\\textwidth"} Another challenge addressed in this work is that Mantis, like most existing TSFMs, is designed for univariate time series, whereas many real-world applications involve observations with multiple time series channels [@wei2018multivariatebook]. A straightforward approach in such cases is to apply a TSFM independently to each channel. However, this method has significant drawbacks when working with datasets containing hundreds or thousands of channels [@bagnall2018uea]. Treating channels independently results in increased runtime and memory consumption, making TSFMs less practical for environments with limited computational resources. To address this limitation, we provide Mantis with several adapters that compress the original channels into a smaller number of new channels, which are then processed by the foundation model. Our experiments demonstrate that these adapters not only substantially improve inference speed but also enable efficient fine-tuning of Mantis for tasks involving a very large number of channels. Methodology {#sec:method} =========== In this section, we present the main technical details behind Mantis: we mathematically introduce the problem setup, discuss the data pre-processing, and present the architecture and the pre-training process. Problem Setup {#sec:problem-setup} ------------- Mathematically speaking, our time series classification foundation model is an encoder $F: \R^t \to \R^{q}$ that projects any time series $\mbf{x}\in\R^t$ with a fixed sequence length $t$ to a discriminative hidden space $\R^{q}$. During the pre-training phase, we observe an unlabeled pre-training set $\mathrm{X}_{\text{0}}$ that is sufficiently large in order to learn rich embeddings that generalize well across different tasks. During the fine-tuning phase, we observe a supervised downstream task with observations $\mathrm{X}$ and labels $\mathrm{Y}$. The two options are possible: 1) we use $F$ to extract deep embeddings $\mathrm{Z}=\{F(\mbf{x}),\ \mbf{x}\in\mathrm{X}\}$ and then learn any classifier $h:\R^{q}\to \{1,\dots,K\}$ using $\mathrm{Z}$ as features and $\mathrm{Y}$ as corresponding labels, 2) append a classification head $h:\R^{q}\to \R^K$ and fine-tune $h\circo F$ by minimizing a loss function evaluated on the downstream dataset. When time series with multiple channels $\mbf{x}=[\mbf{x}_1,\dots,\mbf{x}_d]\in\R^{d\times t},\ d>1$, are considered, we send each channel $\mbf{x}_i,\ i\in[1,d]$, to the TSFM independently, i.e., the embedding of $\mbf{x}$ is defined as $\mbf{z}=\textrm{concat}\left[(F(\mbf{x}_i))_{1\leq i\leq d}\right]$, where $\textrm{concat}$ denotes the vector concatenation operator, and the input dimension of the classifier (head) is $\R^{d\times q}$. As we will discuss in Section `\ref{sec:adapters}`{=latex}, when $d$ is large, we can employ an adapter $a: \R^{d\times t} \to \R^{d_{\text{new}}\times t}$ that compresses the original channels into new $d_{\text{new}}$ channels. In this case, the embedding $\mbf{z}$ is calculated as $\mbf{z}=\textrm{concat}\left[(F(a(\mbf{x})_{i}))_{1\leq i\leq d_{\text{new}}}\right]$, where $a(\mbf{x})_i$ denotes the $i$-th new channel. When it comes to quantifying the uncertainty of a fine-tuned foundation model, we consider its predicted probabilities $\sigma(h\circo F(\mbf{x}))$, with $\sigma: \R^K\to\Delta_K$ being the softmax transformation that projects logits onto the probability simplex $\Delta_K = \{\mbf{p} \in[0, 1]^K\,|\sss\norm{\mbf{p}}_1 = 1\}$. We particularly focus on the top probability $\textrm{conf}(\mbf{x})=\max\left[\sigma(h\circo F(\mbf{x}))\right]$, which we consider the confidence of the model in classifying $\mbf{x}$. For the sake of simplicity, the predicted label we denoted by $\hat{y} = \argmax\left[\sigma(h\circo F(\mbf{x}))\right]$. Data Pre-processing {#sec:preprocessing} ------------------- ```{=latex} \renewcommand{\arraystretch}{1.15} ``` ```{=latex} \centering ``` ```{=latex} \vspace{0.2cm} ``` ::: {#tab:my_label} **Model name** **Multivar Support** **Zero-shot** **HuggingFace Support** **Model Size** ---------------- ---------------------- -------------------- ------------------------- --------------------------- Mantis `\tick `{=latex} `\tick `{=latex} `\tick `{=latex} $\bullet \circ \circ$ UniTS `\tick `{=latex} `\crossi `{=latex} `\crossi `{=latex} $\bullet \circ \circ$ NuTime `\tick `{=latex} `\tick `{=latex} `\crossi `{=latex} $\bullet \circ \circ$ GPT4TS `\tick `{=latex} `\crossi `{=latex} `\crossi `{=latex} $\bullet \bullet \circ$ MOMENT `\crossi `{=latex} `\tick `{=latex} `\tick `{=latex} $\bullet \bullet \bullet$ : Characteristics of different time series foundation models used for classification. For model size, our notations are the following: $\bullet \circ \circ$ is \<10M parameters, $\bullet \bullet \circ$ is in \[10M, 100M\] and $\bullet \bullet \bullet$ is for models having more than 100M parameters. ::: Since time series data may vary in sequence lengths and units of measurement, pre-processing is required for both pre-training and inference. Similar to practices in computer vision, we pre-train the model with fixed input dimensions and resize inputs during inference and fine-tuning. Specifically, we set the input sequence length of Mantis to 512 and resize inputs using PyTorch's interpolation function [@paszke2019pytorch]. To address the issue of varying units of measurement, we apply instance-level standard scaling. For each time series observation (and, in the case of multiple channels, for each individual channel), we subtract the mean and divide by the standard deviation calculated across the time steps. This scaling process is implemented directly within the model architecture, ensuring that inputs are transformed during the forward pass. Architecture {#sec:architecture} ------------ Below, we give details on the architecture of Mantis, which is an adaptation of the Vision Transformer (ViT, `\mycite{dosovitskiy2021vit}`{=latex}) to the time series framework. The entire architecture is depicted in Figure `\ref{fig:mantis-architecture}`{=latex}. #### Token Generator Unit. The first step is to encode a time series into meaningful tokens, which are then passed to a transformer. Our token generation process consists of the following steps: - Before encoding a time series, we perform instance-level normalization, as explained in Section `\ref{sec:preprocessing}`{=latex}. We then split the time series into patches. While @lin2024nutime and @Yuqietal-2023-PatchTST directly split the time series into non-overlapping and overlapping patches, respectively, we achieve this by applying a single convolution layer followed by mean pooling to obtain 32 patches. We set the number of output channels in the convolution to 256, which means each patch represents 256 convolutional features. - Similarly, we generate patches for the time series differential, which is computed by taking the difference between two adjacent timestamps. The main idea behind the differential is to make the time series stationary, thus reducing the influence of trends. In Section `\ref{sec:ablation-study}`{=latex}, we show that incorporating this feature improves the overall performance. - To preserve information about the original unit measurements, we split the raw time series observation (i.e., before instance normalization) into 32 non-overlapping patches, compute the mean and standard deviation for each patch, and encode them using the Multi-Scaled Scalar Encoder [@lin2024nutime]. - Finally, we concatenate all features (from the time series, its differential, and the statistics) and pass them through a linear projector followed by a layer normalization step [@ba2016layernormalization], generating the final 32 tokens with a dimension of 256. ```{=latex} \newpage ``` #### ViT Unit. We then feed the generated tokens into a ViT unit, with the main steps summarized below: - First, we append a class token to the 32 tokens generated in the previous step. As a learnable vector, the class token is introduced to attend to all the other tokens, aggregating information from the entire input into the embedding associated with it. - To incorporate information about the positions of the tokens, we use the classical sinusoidal positional encoding [@vaswani2017transformer]. The position embeddings are summed with the input tokens and fed into a series of transformer layers. - The transformer layer is identical to the one used by @dosovitskiy2021vit. We apply 6 transformer layers, each with multi-head attention consisting of 8 heads. During pre-training, we use a dropout rate of $10\pct$. - Finally, the class token's representation generated by the transformer serves as the final output of the foundation model. #### Projector and Prediction Head. To use the output of the foundation model, we append different layers depending on whether the model is in the pre-training or fine-tuning stage: - *Pre-training:* At this stage, we add a layer normalization step followed by a linear layer, which projects the embeddings for calculating their similarities. - *Fine-tuning:* At this stage, we add a classification head that maps the embeddings to class logits. Pre-training {#sec:pre-training} ------------ We pre-train Mantis in a self-supervised way using a contrastive learning approach that aims to train an encoder that outputs similar representations for two random augmentations of the same sample (positive pair) and dissimilar representations for augmentations of two different samples (negative pair). More formally, let $\mathcal{T}$ be a considered space of transformations (augmentations) such that $\forall\phi\in\mathcal{T},\mbf{x}\in\mathcal{X}$ we have $\phi(\mbf{x})\in\mathcal{X}$. To measure the similarity of two embeddings, we first project the output of the foundation model $F(\mbf{x})$ to a new dimension using a projector $g: \R^{q} \to \R^{q'}$ and then compute the cosine similarity between the two vectors defined as follows: $$\begin{aligned} s_{\cos}(\mathbf{a}, \mathbf{b}) := \frac{\mbf{a}^\top\mbf{b}}{\norm{\mbf{a}}\cdot\norm{\mbf{b}}},\qquad \forall(\mbf{a}, \mbf{b})\in\R^{2q'}.\end{aligned}$$ Given a batch $B=\{\mbf{x}_i\}_{i=1}^b$, for each example $\mbf{x}_i$, we sample two augmentation functions $\phi$ and $\psi$ uniformly from $\mathcal{T}$, i.e., $\phi,\psi\sim\mathcal{U}(\mathcal{T})$, compute the pairwise similarities between all the examples in the following way: $$\begin{aligned} \mbf{s}_i(\phi, \psi) = \left[s_{\cos}\left(g\circo F\circo\phi(\mbf{x}_i), \, g\circo F\circo\psi(\mbf{x}_j)\right)\right]_{j=1}^b \in \R^b.\end{aligned}$$ Following @oord2018representation as well as @he2020momentum and denoting the cross-entropy error function by $l_{\text{ce}}: \R^b\times\{1,\dots,b\}\to\R$, we update the weights of $F$ and $g$ by minimizing the contrastive loss which we define as $$\begin{aligned} \sum_{i=1}^b l_{\text{ce}}\left(\frac{\mbf{s}_i(\phi, \psi)}{T},\ i\right),\end{aligned}$$ where $T\in(0,+\infty)$ is a temperature that we fixed to $0.1$. `\setlength{\intextsep}{-5pt}`{=latex} `\setlength{\columnsep}{10pt}`{=latex} ```{=latex} \begin{wrapfigure}[11]{r}{0.27\textwidth} \includegraphics[width=0.26\textwidth, clip=True, trim=0 0 0 0]{pics/randomcropresize.png} \caption{\texttt{RandomCropResize}} \label{fig:randomcropresize} \end{wrapfigure} ``` Regarding the choice of augmentation methods, we empirically tested several augmentation functions and found that their effectiveness is highly dataset-dependent, as they may distort a time series and cause the loss of important information. For our pre-training, we chose the `RandomCropResize` augmentation (Figure `\ref{fig:randomcropresize}`{=latex}), which involves randomly cropping $c\pct$ of a time series (with the condition that the remaining $(1\!-\!c)\pct$ forms a contiguous window) and then resizing the cropped portion to the original sequence length. We applied relatively moderate distortions, varying the crop rate from $0\pct$ to $20\pct$, preserving the main structure of a time series. As a pre-training dataset, we use a union of various public datasets including UCR [@dau2019ucr], UEA [@bagnall2018uea all except EigenWorms and InsectWingbeat], ECG [@clifford2017ecgdataset], EMG [@goldberger2000emgdataset], Epilepsy [@andrzejak2001epilepsydataset], FD-A and FD-B [@lessmeier2016fdafdbdataset], Gesture [@liu2009gesturedataset], HAR [@anguita2013hardataset], SleepEEG [@kemp2000sleepeegdataset]. We ensure that test sets used for evaluation in Section `\ref{sec:experiments}`{=latex} are not part of the pre-training dataset. The total pre-training consists of 7 million time series examples. We pre-trained the model for 100 epochs with a batch size equal to 2048 on 4 NVIDIA Tesla V100-32GB GPUs. Adapters {#sec:adapters} -------- Nowadays, the multivariate setting represents one of the main challenges for time series foundation models. In practice, different time series classification tasks vary in the number of channels, which raises non-trivial questions on how the model should be pre-trained and applied for a new task. Similarly to other foundation models [@goswami2024moment; @lin2024nutime], Mantis is pre-trained in a univariate way and can be applied to the multivariate setting by treating each channel independently. However, this strategy has several limitations. Firstly, the foundation models are heavy by design, so treating all channels independently may lead to excessive resource consumption and failure to fine-tune them. Secondly, at the feature extraction step, the channel correlations are ignored, which may lead to the loss of some important information. To address these concerns, we study a simple approach of using a channel-level adapter $a: \R^{d} \to \R^{d_{\text{new}}}$ that precedes the foundation model and transforms the original $d$ channels into new $d_{\text{new}}$ ones. While there are other ways to adapt foundation models to the multivariate setting [@zhang2023crossformer; @zhou2023onefitsall], we consider this definition of an adapter due to its high flexibility: (a) not only Mantis but any foundation model can be plugged in, (b) channel-level transformation prevents from disrupting the temporal structure, (c) adaptation to the computation budget by determining the number of encoded channels. In our experiments, we considered five adapters that we present further. Our idea was to test classical dimension reduction approaches like Principal Component Analysis (PCA) that are computationally efficient and are trained in an unsupervised way. However, their application is limited to 2D data matrices, making them not straightforward to use in our case. To overcome this problem, we train the dimension reduction method on the data reshaped to $(n\times t,\ d)$, where $n$ is the number of training examples. Thus, with this trick, methods like PCA will be focused on correlations between channels over all time steps, capturing spatial correlations and learning the rotation matrix $W \in \R^{d_{\text{new}}\times d}$ that linearly combines the original channels into new ones. Overall, these four adapters rely on this strategy: - *Principal Component Analysis* (*PCA*) seeks to find an orthogonal basis of principal components where few components capture most of the data variance. - *Truncated Singular Value Decomposition* (*SVD*, @halko2009truncatedsvd, [-@halko2009truncatedsvd]) is similar to PCA, but it applies SVD decomposition directly on the non-centered data matrix. - *Random Projection (Rand Proj)* is a very simple baseline that projects data by randomly generating the rotation matrix $W \in \R^{d_{\text{new}}\times d}$. - *Variance-Based Channel Selection (Var Selector)* is a feature selection method that selects channels with the highest variance. This approach can be useful when channels with low variance are not very informative, so they can be discarded without affecting the overall performance. Although these adapters are computationally very efficient, they are not supervised meaning that they may lead to suboptimal classification performance. Following this reasoning, we introduce one more adapter: - *Differentiable Linear Combiner* (*LComb*) performs a linear channel transformation $W \in \mathbb{R}^{D' \times D}$, which, unlike to previous adapters, is learned via backpropagation together with the encoder and classification head. Experimental Results {#sec:experiments} ==================== In this section, we conduct a set of experiments to validate the high performance of our model. All our experiments have been performed on a single NVIDIA Tesla V100-32GB GPU. Setup {#sec:exp-setup} ----- #### Baselines. In our experiments, we compare Mantis with the State-of-the-Art (SOTA) time series classification foundation modeling. More specifically, these are the four following foundation models, whose implementation details can be found in Appendix `\ref{sec:appendix-exp-setup}`{=latex}: - UniTS [@gao2024units] is a multi-task time series model that can be also considered as a foundation model with 1 million parameters. We use their checkpoint pre-trained in a supervised fashion on 18 datasets. - GPT4TS [@zhou2023onefitsall] is a method that partially fine-tunes a pre-trained GPT2 with an additional tokenization layer to perform different time series tasks including classification, anomaly detection, and forecasting. The total number of parameters is approximately 80 millions (10x larger than Mantis). - NuTime [@lin2024nutime] is a transformer-based TSFM for classification. Their pre-training dataset is very similar to the one we used, and the size of their model is smaller, containing approximately 2 million parameters. - MOMENT [@goswami2024moment] is a TSFM based on the T5 architecture [@raffel2020exploring] that has been pre-trained to cover different time series tasks including classification, anomaly detection, and forecasting. In total, MOMENT has 385 million parameters (approx. 48x larger than Mantis) and the pre-training dataset contained 1.13 billion samples. #### Datasets. We use the following datasets to evaluate the performance of Mantis and compare it to other methods: - UCR [@dau2019ucr] that consists of 128 single-channel datasets. - UEA [@bagnall2018uea] that has 30 multi-channel datasets. We exclude 3 datasets due to small test size or small sequence length, and subsample one dataset to ease computations (see more details in Appendix `\ref{sec:appendix-exp-setup}`{=latex}). We further tag the remaining 27 datasets by `UEA-27`. - 4 additional datasets used by @gao2024units: Blink [@chicaiza2021blink], MotionSenseHAR [@malekzadeh2019motionsensehar], EMOPain [@egede2020emopain], SharePriceIncrease [@middlehurst2024bakeoff]. #### Experiments. We have performed five different kinds of experiments, which we will briefly describe below. For all experiments except for the adapters, Mantis handles the multivariate case by sending all channels to the encoder independently. 1. *Zero-shot feature extraction.* In this experiment, we test the ability of Mantis to generate powerful embeddings without fine-tuning. For each downstream task, we extract features using the foundation model and then use them together with the training labels to learn a classifier. We use all considered datasets: the whole UCR collection, `UEA-27`, and 4 datasets from @gao2024units, which results in 159 datasets (further denoted by `159-D`). 2. *Fine-tuning with model selection.* For this experiment, we compare different foundation models when they are fine-tuned to a given downstream task. For each task, we use 80% of the original training set to fine-tune models and the rest 20% as a validation set to choose training hyperparameters. As we have fixed the batch size for all the considered methods, some methods did not fit a single GPU memory either due to the number of channels or the sequence length. This is why we removed from the results 15 datasets from UCR, 11 datasets from UEA, EMOPain, and MotionSenseHAR (see more details in Appendix `\ref{sec:appendix-exp-setup}`{=latex}), so in total, we evaluate on 131 datasets (further tagged by `131-D`). 3. *Ablation study.* In this experiment, we analyze the influence of different parts of the proposed methodology on the performance of Mantis. First, we show the advantage of extracting features from the time series' differential for zero-shot extraction. Then, we analyze the influence of pre-training, fine-tuning, and model selection on the performance. We consider all the datasets except those with a large number of channels: 7 datasets from UEA and EMOPain. In total, it results in 151 datasets (further denoted by `151-D`). 4. *Adapters.* We perform a comparative study of different adapters that extend Mantis to the large-dimensional case. In this experiment, consider all the `UEA-27` datasets described before. 5. *Calibration.* Finally, we compare Mantis with other foundation models in terms of calibration of their predicted probabilities. We take the same experimental setup as for the *fine-tuning with model selection* experiment and, for each model, report the Expected Calibration Error (ECE, @naeini2015obtaining, [-@naeini2015obtaining]) before and after applying post-hoc calibration methods. Comparison with SOTA: Zero-shot Feature Extraction {#sec:sota-zero-shot} -------------------------------------------------- In this section, we use a time series foundation model as a feature extractor without fine-tuning it to a given downstream task. First, we generate embeddings for time series observations by passing them through the frozen encoder. Then, we use embeddings as new features to train a classifier and report its performance on test data. As a classifier, we have chosen the Random Forest algorithm [@breiman2001random] due to its versatility and high performance without the need to perform hyperparameter searching. We fixed the number of trees to 200 while not restricting the maximum tree depth. In this section, we compare Mantis with NuTime and MOMENT while removing GPT4TS and UniTS from consideration as they have not provided a methodology for zero-shot extraction. `\setlength{\intextsep}{-5pt}`{=latex} `\setlength{\columnsep}{10pt}`{=latex} ```{=latex} \begin{wrapfigure}[18]{r}{0.4\textwidth} \centering \vspace{-0.21cm} \includegraphics[height=0.25\textheight, clip=True, trim=0 0.5cm 0 0]{pics/159-d-zeroshot-perf-results.pdf} \caption{Zero-shot feature extraction: comparison with the SOTA. The accuracy is averaged over 3 random seeds and over \texttt{159-D} datasets.} \label{fig:zeroshot-sota} \end{wrapfigure} ``` Figure `\ref{fig:zeroshot-sota}`{=latex} displays the performance results averaged over 159 datasets (`159-D`), while the complete results can be found in Appendix, Table `\ref{tab:zeroshot-sota}`{=latex}. We can see that on average Mantis outperforms NuTime and MOMENT by 1.77% and 1.53%, respectively. Mantis is the best on 74 datasets, while MOMENT and NuTime have the best performance on 53 and 38 datasets, respectively. Compared to other foundation models, we can see that Mantis achieves the best balance in terms of the complexity of the model and its accuracy. On the one hand, Mantis outperforms a smaller model, NuTime, by increasing the encoder's capacity and leveraging more information from the pre-training set. On the other hand, a much bigger model, MOMENT, with 385 million parameters does not manage to achieve a better performance despite its size. In addition, comparing models in terms of running time, we notice that the forward pass of MOMENT is noticeably slower, which is especially pronounced with a large number of channels. To illustrate this observation, Figure `\ref{fig:runtime-rf}`{=latex} depicts the running time of Mantis and MOMENT on datasets from `UEA-27`. One can see that the inference time of MOMENT greatly increases with the dataset size. ```{=latex} \vspace{0.5cm} ``` ```{=latex} \centering ``` ![The running time of MOMENT and Mantis on the `UEA-27` collection of datasets. For better visualization, we sort and split the datasets into two clusters based on the reported running time.](pics/run-time-rf.png){#fig:runtime-rf width="95%"} ```{=latex} \vspace{0.5cm} ``` Comparison with SOTA: Fine-tuning {#sec:sota-fine-tuning} --------------------------------- `\setlength{\intextsep}{-5pt}`{=latex} `\setlength{\columnsep}{10pt}`{=latex} ```{=latex} \begin{wrapfigure}[15]{r}{0.5\textwidth} \centering \vspace{-0.55cm} \includegraphics[width=0.48\textwidth, clip=True, trim=0 0.5cm 0 0]{pics/131-d-finetuning-perf-results.pdf} \caption{Model fine-tuning: comparison with the SOTA. The accuracy is averaged over 3 random seeds and over \texttt{131-D} datasets.} \label{fig:sota-fine-tuning} \end{wrapfigure} ``` In this experiment, we perform a comparison by fine-tuning foundation models to a downstream task. More specifically, we append a prediction head after an encoder and fine-tune all layers or a subset of them on the training data. For all models under consideration, we fix a fine-tuning scheme: we minimize the cross-entropy loss for 100 epochs with a fixed batch size equal to 256, using an AdamW optimizer [@loshchilov2017fixing] with a weight decay of 0.05. For each model and each dataset, We choose a learning rate over the grid: $\{10^{-4}, 2\cdot10^{-4}, 10^{-3}\}$ based on the validation set. For GPT4TS, we follow their paper and fine-tune the layers of the language backbone specified by the authors. For Mantis, NuTime, MOMENT, and UniTS, we perform fine-tuning of all the layers. In Figure `\ref{fig:sota-fine-tuning}`{=latex}, we illustrate the performance results averaged over 131 datasets (`131-D`). In Appendix, Table `\ref{tab:sota-fine-tuning}`{=latex}, we display the experimental results for every dataset. One can see that Mantis has the highest performance among all the considered foundation models. We note that the reported performance results of our competitors may diverge from those reported by the authors of these models. This may be explained by the difference in the chosen fine-tuning scheme. In our case, we have fixed the scheme for a fair comparison while trying to find the best learning rate for every model based on a validation set. Thus, our experimental results reveal that Mantis is the most robust to the choice of a fine-tuning scheme and does not necessarily require tedious hyperparameter searching. We additionally validate this argument in the next section by fine-tuning Mantis without any model selection. Ablation Study {#sec:ablation-study} -------------- `\setlength{\intextsep}{-5pt}`{=latex} `\setlength{\columnsep}{10pt}`{=latex} ```{=latex} \begin{wrapfigure}[18]{r}{0.35\textwidth} \centering \includegraphics[height=0.25\textheight, clip=True, trim=0 0 0 0]{pics/159-d-ablation-perf-results.pdf} \caption{Mantis w/ and w/o the differential block. The accuracy is averaged over 3 random seeds and over \texttt{159-D} datasets.} \label{fig:zeroshot-ablation} \end{wrapfigure} ``` In this section, we perform several ablation experiments for a more thorough analysis of Mantis. First, we validate some of our architecture choices. More specifically, we test whether the incorporation of features extracted from the differential of a time series, proposed in Section `\ref{sec:architecture}`{=latex}, improves the quality of embeddings. For this, we have pre-trained another version of our foundation model where the differential feature extraction part was removed from the Token Generator Unit. In Figure `\ref{fig:zeroshot-ablation}`{=latex}, we plot the average zero-shot feature extraction performance of the two versions of Mantis, while the complete results may be found in Appendix, Table `\ref{tab:ablation-diff}`{=latex}. One can see that the incorporation of the differential block noticeably improves the accuracy score. When comparing Figure `\ref{fig:zeroshot-sota}`{=latex} and Figure `\ref{fig:zeroshot-ablation}`{=latex}, it is interesting to notice that Mantis without the differential block still slightly outperforms NuTime and MOMENT in the case of zero-shot feature extraction. In the second part of our ablation study, we raise the following questions: (a) is pre-training really useful, or it is sufficient to train Mantis from scratch on each dataset?, (b) should we fine-tune the head or the whole model?, (c) can Mantis be fine-tuned with default hyperparameters, and how much we can gain from the model selection? To answer these questions, we empirically compare the following fine-tuning regimes: - The setup which we further call RF, and which we used for the zero-shot feature extraction experiment: the encoder of Mantis is frozen, and embeddings are used as features to train a Random Forest classifier. - The encoder of Mantis is frozen, and the task is solved by fine-tuning a classification head, which, in our case, is layer normalization step + linear layer. The purpose of comparing this strategy, further called `Head`, with `RF` is to see the impact of the choice of a classifier (linear vs non-linear, differentiable vs non-differentiable) on the overall performance. - The encoder is randomly initialized and fine-tuned together with a classification head. This baseline, which we further call `Scratch`, is introduced to see whether or not pre-training of Mantis is useful. - Finally, the strategy further called `Full` consists in fine-tuning the pre-trained encoder together with a classification head. In order to answer question (c), for every method, we report two scores: the test accuracy of the model at the last epoch (i.e., at the end of fine-tuning), and the best test accuracy of the model across all fine-tuning epochs. While the last epoch score gives a pessimistic evaluation of the performance as no model selection is performed, the best epoch score is an optimistic evaluation as it indicates the best achievable performance. In this experiment, we fix the fine-tuning scheme: the number of epochs is 100, the batch size is 256, the optimizer is AdamW with the weight decay of 0.05 and the learning rate set to $2\cdot10^{-4}$ and adjusted following the cosine annealing decay strategy [@loshchilov2016sgdr] with 10 warm-up epochs. Figure `\ref{fig:fine-tuning-ablation}`{=latex} depicts the average performance over the `151-D` benchmark both for the last and best epoch, while the complete results can be found in Appendix, Table `\ref{tab:last-epoch-res}`{=latex} and Table `\ref{tab:best-epoch-res}`{=latex}. First, we can see that the pre-training of Mantis is visibly useful as `Full` outperforms `Scratch` by 5.08% and 3.92% in the last and best epoch score, respectively. On the other hand, using a frozen pre-trained encoder is rather suboptimal in terms of performance, and full fine-tuning is preferred. This observation points out an important direction of future work, namely, to advance time series classification foundation models in terms of zero-shot performance. ```{=latex} \centering ``` ![Performance of different fine-tuning methods for Mantis averaged over 3 random seeds and 151 datasets of the `151-D` benchmark.](pics/151d-perf-results-fine-tuning-last.png){#fig:fine-tuning-ablation width="90%"} ```{=latex} \hfill ``` ```{=latex} \centering ``` ![Performance of different fine-tuning methods for Mantis averaged over 3 random seeds and 151 datasets of the `151-D` benchmark.](pics/151d-perf-results-fine-tuning-best.png){#fig:fine-tuning-ablation width="90%"} `\setlength{\intextsep}{-5pt}`{=latex} `\setlength{\columnsep}{10pt}`{=latex} ```{=latex} \begin{wrapfigure}[16]{r}{0.25\textwidth} % \vspace{-0.8cm} \centering \includegraphics[width=0.21\textwidth, clip=True, trim=0 0.3cm 0 0]{pics/uea-27-perf-results-best.pdf} \caption{Mantis: zero-shot vs full fine-tuning with the best adapter per dataset.} \label{fig:rf-vs-best-adapter} \end{wrapfigure} ``` Another observation from Figure `\ref{fig:fine-tuning-ablation}`{=latex} is that fine-tuning the head with a frozen encoder may be suboptimal, and training a Random Forest on embeddings gives a slightly better accuracy score. Finally, by comparing the performance results for the last and best epoch, we conclude that although model selection is important to improve the overall performance (+1.89% for `Full`), using directly the model from the last epoch gives already good performance. This shows that Mantis can be applied in practice without the need to tediously search for optimal values of hyperparameters. Adapters {#sec:exp-adapters} -------- In this section, we study the performance of Mantis when it is combined with one of the adapters we introduced in Section `\ref{sec:adapters}`{=latex}. For `PCA`, `SVD`, `Rand Proj` and `Var Selector`, the new number of channels is equal to $D'=\min\{D, 10\}$, while for LComb is always fixed to $D'=10$. Table `\ref{tab:adapters-perf}`{=latex} depicts the empirical comparison between the adapters and the case when we treat all channels independently (dubbed as `No Adapter`). In addition, Figure `\ref{fig:rf-vs-best-adapter}`{=latex} and Table `\ref{tab:rf-vs-best-adapter}`{=latex} of the Appendix compare the no-fine-tuning case when Random Forest is used directly after the feature extraction step and the full fine-tuning with the best adapter chosen per dataset. Below, we discuss the experimental results. `\scalebox{0.72}{ \begin{tabular}{l| l || l llll l||l} \toprule & \multirow{2}{*}{d} & \multirow{2}{*}{\texttt{No Adapter}} & \multicolumn{4}{c}{Standalone Adapter} & Diff. Adapter & \multirow{2}{*}{Best Result} \vspace{-0.2cm} \\ & & & \multicolumn{4}{l}{\rule{9.2cm}{0.4pt}} & \rule{2.3cm}{0.4pt} \\ & & & \texttt{PCA} & \texttt{SVD} & \texttt{Rand Proj} & \texttt{Var Selector} & \texttt{LComb} & \\ \midrule ArticularyWordRecognition & 9 & \textbf{0.9933}$_{\pm 0.0}$ & 0.9922$_{\pm 0.0019}$ & 0.9878$_{\pm 0.0038}$ & 0.9811$_{\pm 0.0038}$ & \textbf{0.9933}$_{\pm 0.0}$ & 0.9744$_{\pm 0.0069}$ & 0.9933$_{\pm 0.0}$ \\ BasicMotions & 6 & \textbf{1.0}$_{\pm 0.0}$ & \textbf{1.0}$_{\pm 0.0}$ & \textbf{1.0}$_{\pm 0.0}$ & \textbf{1.0}$_{\pm 0.0}$ & \textbf{1.0}$_{\pm 0.0}$ & \textbf{1.0}$_{\pm 0.0}$ & 1.0$_{\pm 0.0}$ \\ CharacterTrajectories & 3 & 0.9928$_{\pm 0.0004}$ & \textbf{0.9947}$_{\pm 0.0004}$ & 0.9923$_{\pm 0.0012}$ & 0.9912$_{\pm 0.0016}$ & 0.9928$_{\pm 0.0004}$ & 0.993$_{\pm 0.0018}$ & 0.9947$_{\pm 0.0004}$ \\ Cricket & 6 & \textbf{1.0}$_{\pm 0.0}$ & 0.9861$_{\pm 0.0}$ & 0.9769$_{\pm 0.008}$ & 0.9722$_{\pm 0.0}$ & \textbf{1.0}$_{\pm 0.0}$ & 0.9907$_{\pm 0.008}$ & 1.0$_{\pm 0.0}$ \\ DuckDuckGeese & 1345 & \texttt{NaN} & 0.5733$_{\pm 0.0115}$ & \textbf{0.6}$_{\pm 0.02}$ & 0.54$_{\pm 0.0872}$ & 0.5133$_{\pm 0.0231}$ & 0.54$_{\pm 0.0693}$ & 0.6$_{\pm 0.02}$ \\ ERing & 4 & \textbf{0.9926}$_{\pm 0.0074}$ & 0.9778$_{\pm 0.0064}$ & 0.9753$_{\pm 0.0113}$ & 0.9642$_{\pm 0.0043}$ & \textbf{0.9926}$_{\pm 0.0074}$ & 0.9778$_{\pm 0.0064}$ & 0.9926$_{\pm 0.0074}$ \\ EigenWorms & 6 & 0.8372$_{\pm 0.0044}$ & 0.8448$_{\pm 0.0117}$ & \textbf{0.8601}$_{\pm 0.0192}$ & 0.8117$_{\pm 0.0233}$ & 0.8372$_{\pm 0.0044}$ & 0.8066$_{\pm 0.0384}$ & 0.8601$_{\pm 0.0192}$ \\ Epilepsy & 3 & \textbf{1.0}$_{\pm 0.0}$ & 0.9976$_{\pm 0.0042}$ & 0.9976$_{\pm 0.0042}$ & 0.9976$_{\pm 0.0042}$ & \textbf{1.0}$_{\pm 0.0}$ & \textbf{1.0}$_{\pm 0.0}$ & 1.0$_{\pm 0.0}$ \\ EthanolConcentration & 3 & \textbf{0.4208}$_{\pm 0.0195}$ & 0.2928$_{\pm 0.0101}$ & 0.3029$_{\pm 0.0122}$ & 0.384$_{\pm 0.0503}$ & \textbf{0.4208}$_{\pm 0.0195}$ & 0.4081$_{\pm 0.0275}$ & 0.4208$_{\pm 0.0195}$ \\ FaceDetection & 144 & \texttt{NaN} & 0.6026$_{\pm 0.0054}$ & 0.6064$_{\pm 0.0037}$ & 0.5638$_{\pm 0.0065}$ & 0.5696$_{\pm 0.0027}$ & \textbf{0.6272}$_{\pm 0.013}$ & 0.6272$_{\pm 0.013}$ \\ FingerMovements & 28 & \texttt{NaN} & 0.5833$_{\pm 0.0058}$ & 0.57$_{\pm 0.06}$ & 0.5467$_{\pm 0.0289}$ & \textbf{0.6167}$_{\pm 0.0058}$ & 0.58$_{\pm 0.0265}$ & 0.6167$_{\pm 0.0058}$ \\ HandMovementDirection & 10 & 0.4009$_{\pm 0.0206}$ & \textbf{0.5135}$_{\pm 0.027}$ & 0.482$_{\pm 0.0546}$ & 0.4279$_{\pm 0.0512}$ & 0.4009$_{\pm 0.0206}$ & 0.4414$_{\pm 0.0624}$ & 0.5135$_{\pm 0.027}$ \\ Handwriting & 3 & 0.482$_{\pm 0.0157}$ & 0.4529$_{\pm 0.0129}$ & 0.4588$_{\pm 0.0224}$ & 0.4235$_{\pm 0.0418}$ & 0.482$_{\pm 0.0157}$ & \textbf{0.5839}$_{\pm 0.0283}$ & 0.5839$_{\pm 0.0283}$ \\ Heartbeat & 61 & \texttt{NaN} & 0.7561$_{\pm 0.0098}$ & 0.7626$_{\pm 0.0123}$ & 0.7707$_{\pm 0.0098}$ & \textbf{0.7951}$_{\pm 0.0129}$ & 0.774$_{\pm 0.0123}$ & 0.7951$_{\pm 0.0129}$ \\ InsectWingbeatSubset & 200 & \texttt{NaN} & 0.4703$_{\pm 0.0051}$ & 0.4733$_{\pm 0.0127}$ & 0.4803$_{\pm 0.0302}$ & \textbf{0.591}$_{\pm 0.005}$ & 0.236$_{\pm 0.0125}$ & 0.591$_{\pm 0.005}$ \\ JapaneseVowels & 12 & \textbf{0.9811}$_{\pm 0.0054}$ & 0.9802$_{\pm 0.0031}$ & \textbf{0.9811}$_{\pm 0.0}$ & 0.9577$_{\pm 0.0271}$ & 0.9784$_{\pm 0.0047}$ & 0.9577$_{\pm 0.0128}$ & 0.9811$_{\pm 0.0054}$ \\ LSST & 6 & \textbf{0.7109}$_{\pm 0.0015}$ & 0.6795$_{\pm 0.0027}$ & 0.6894$_{\pm 0.0021}$ & 0.6929$_{\pm 0.0207}$ & \textbf{0.7109}$_{\pm 0.0015}$ & 0.6941$_{\pm 0.0053}$ & 0.7109$_{\pm 0.0015}$ \\ Libras & 2 & 0.9389$_{\pm 0.0}$ & 0.937$_{\pm 0.0032}$ & 0.9481$_{\pm 0.0064}$ & 0.8111$_{\pm 0.1392}$ & 0.9389$_{\pm 0.0}$ & \textbf{0.9704}$_{\pm 0.0032}$ & 0.9704$_{\pm 0.0032}$ \\ MotorImagery & 64 & \texttt{NaN} & 0.5933$_{\pm 0.0351}$ & 0.5833$_{\pm 0.0208}$ & 0.5867$_{\pm 0.0379}$ & \textbf{0.6}$_{\pm 0.01}$ & 0.59$_{\pm 0.0173}$ & 0.6$_{\pm 0.01}$ \\ NATOPS & 24 & 0.937$_{\pm 0.0116}$ & 0.9537$_{\pm 0.0064}$ & \textbf{0.9611}$_{\pm 0.0}$ & 0.8926$_{\pm 0.0032}$ & 0.8981$_{\pm 0.0116}$ & 0.8796$_{\pm 0.017}$ & 0.9611$_{\pm 0.0}$ \\ PEMS-SF & 963 & \texttt{NaN} & 0.8536$_{\pm 0.0067}$ & 0.8304$_{\pm 0.0145}$ & 0.7457$_{\pm 0.0473}$ & \textbf{0.9114}$_{\pm 0.0067}$ & 0.7476$_{\pm 0.0219}$ & 0.9114$_{\pm 0.0067}$ \\ PhonemeSpectra & 11 & 0.3421$_{\pm 0.0023}$ & 0.3351$_{\pm 0.009}$ & 0.3215$_{\pm 0.0052}$ & 0.3492$_{\pm 0.0033}$ & 0.344$_{\pm 0.0052}$ & \textbf{0.3547}$_{\pm 0.0047}$ & 0.3547$_{\pm 0.0047}$ \\ RacketSports & 6 & \textbf{0.9408}$_{\pm 0.0}$ & 0.9123$_{\pm 0.0152}$ & 0.9101$_{\pm 0.0076}$ & 0.9167$_{\pm 0.0038}$ & \textbf{0.9408}$_{\pm 0.0}$ & 0.9254$_{\pm 0.0038}$ & 0.9408$_{\pm 0.0}$ \\ SelfRegulationSCP1 & 6 & 0.9135$_{\pm 0.0071}$ & \textbf{0.917}$_{\pm 0.0052}$ & 0.9113$_{\pm 0.0034}$ & 0.9135$_{\pm 0.0079}$ & 0.9135$_{\pm 0.0071}$ & 0.901$_{\pm 0.009}$ & 0.917$_{\pm 0.0052}$ \\ SelfRegulationSCP2 & 7 & 0.5389$_{\pm 0.0096}$ & 0.5648$_{\pm 0.0449}$ & \textbf{0.5685}$_{\pm 0.0251}$ & 0.5611$_{\pm 0.0455}$ & 0.5389$_{\pm 0.0096}$ & 0.5482$_{\pm 0.021}$ & 0.5685$_{\pm 0.0251}$ \\ SpokenArabicDigits & 13 & 0.987$_{\pm 0.0009}$ & \textbf{0.9933}$_{\pm 0.0014}$ & 0.9906$_{\pm 0.0019}$ & 0.988$_{\pm 0.0027}$ & 0.9864$_{\pm 0.0028}$ & 0.9882$_{\pm 0.0016}$ & 0.9933$_{\pm 0.0014}$ \\ UWaveGestureLibrary & 3 & \textbf{0.9438}$_{\pm 0.0108}$ & 0.8583$_{\pm 0.0079}$ & 0.8552$_{\pm 0.0095}$ & 0.8635$_{\pm 0.0737}$ & \textbf{0.9438}$_{\pm 0.0108}$ & 0.9177$_{\pm 0.0048}$ & 0.9438$_{\pm 0.0108}$ \\ % \midrule % \textit{\textbf{Best Count}} & & 10 \textbf{109} \\ \bottomrule \end{tabular} }`{=latex} `\label{tab:adapters-perf}`{=latex} *Dimension reduction:* When the number of channels is too large, the full fine-tuning of the foundation model is limited as treating all channels independently is computationally costly. In Table `\ref{tab:adapters-perf}`{=latex}, one can see that starting from 28 channels and the batch size fixed to 256, the optimization of Mantis does not fit the memory of a single V100-32GB GPU card, which results in 7 `NaN` values in the `No Adapter` column. Nevertheless, applying one of the five adapters solves the issue, allowing the full fine-tuning and increasing the performance as can be seen in Table `\ref{tab:rf-vs-best-adapter}`{=latex}. *Channel interactions:* Another advantage of adapters is that they mix the original channels allowing channels to interact before they are sent to the encoder. The utility of adapters from this perspective was shown by @benechehab2024zero for zero-shot multivariate forecasting. Our experiments reveal that, in the classification context, conclusions are rather dataset-dependent: while for datasets like RacketSports and UWaveGestureLibrary, adapters do not bring any improvement compared `No Adapter`, transforming the original channels for datasets like Handwriting and HandMovementDirection, nevertheless, brings a large improvement. *Channel selection:* In some cases certain channels do not contain any useful information, so they can simply be discarded. In such situations, the `Var Selector` has the highest performance, which we can observe on FingerMovements and InsectWingbeat datasets. *Differentiable adapter:* Although on some datasets, `LComb` yields high performance, overall, its performance is lower than those of `PCA`, `SVD` and `Var Selector`. Since `LComb` is fine-tuned together with the encoder and the head, we suggest that the optimization of the adapter is intricate, so searching for a better optimization scheme can be a good direction for future work. Calibration {#sec:exp-calibration} ----------- Uncertainty quantification is crucial when a learning model is part of a safety-critical system [@wang2023calibration] or when its output is used to analyze the performance on test data [@xie2024mano]. Consequently, in this section, we study the calibration properties of Mantis and other time series classification foundation models. We say a model is calibrated if, for a given confidence level $\alpha$, the probability that the model predicts the true class is equal to $\alpha$ [@guo2017calibration]: $$\mathbb{P}(Y = \hat{y} \,|\, \textrm{conf}(\mbf{X}) = \alpha) = \alpha, \quad \forall \alpha\in[0, 1],$$ where $\hat{y}$ and $\textrm{conf}(\mbf{X})$ denote the predicted label and model's confidence of a random variable $\mbf{X}$, respectively, which were formally defined in Section `\ref{sec:problem-setup}`{=latex}. We consider the Expected Calibration Error (ECE, @naeini2015obtaining, [-@naeini2015obtaining]) as a metric to evaluate the calibration on a test set $\{\mbf{x}_i, y_i\}_{i=1}^n$. We discretize the interval $[0,1]$ into $m$ disjoint bins: $B_j\cap B_k=\emptyset,\ \forall j\neq k, 1 \leq j,k\leq m$ and $\cup_{j=1}^m B_j=[0,1]$. As commonly done in the literature, we split the interval into 10 equally spaced bins. Then, denoting the predicted label for $\mbf{x}_i$ by $\hat{y}_i$, the ECE is defined as follows: $$\text{ECE} = \frac{1}{n} \sum_{j=1}^{m} \left| \sum_{\substack{{1\leq i\leq n}\\{\textrm{conf}(\mbf{x}_i)\in B_j }}} \I{\hat{y}_i=y_i}- \textrm{conf}(\mbf{x}_i)\,\right|.$$ Following the setup in Section `\ref{sec:sota-fine-tuning}`{=latex}, we evaluate the ECE before and after applying a *post-hoc* calibration for the five fine-tuned foundation models on 131 datasets from `131-D`. We use two calibration techniques: (a) Temperature Scaling [@guo2017calibration] that regulates the amplitude of logits before applying softmax, and (b) Isotonic Regression [@zadrozny2002transforming], a non-parametric method that adjusts predicted probabilities by fitting a piecewise constant function. For the second approach, instead of the classical algorithm, we consider the multi-class isotonic regression (IRM, @zhang2020mix, [-@zhang2020mix]) as it is shown to be less prone to overfitting. For each dataset, both approaches are fitted using the corresponding validation set (20% of training data as described in Section `\ref{sec:exp-setup}`{=latex}). In Figure `\ref{fig:avg-calibration}`{=latex}, the Expected Calibration Error is displayed for the five foundation models under consideration before and after applying a post-hoc calibration method. We can observe that Mantis without any post-hoc calibration is the most calibrated model on average over 131 datasets, which is an important property for those applications where the number of training examples is not sufficient to construct a hold-out validation set. When posthoc calibration is applied, using either the Isotonic Regression or the Temperature Scaling, all foundation models become significantly better calibrated, yet Mantis remains the most calibrated. ```{=latex} \centering ``` ![Averaged expected calibrated error of different methods over `131-D` datasets.](pics/131-d-finetuning-ece-all-results.png){#fig:avg-calibration width="\\textwidth"} ```{=latex} \vspace{0.5cm} ``` ```{=latex} \centering ``` ![The reliability diagram on the test set after *post-hoc* temperature scaling on the validation set, for three different datasets. The gray histogram illustrates the distribution of the confidence score across the test set, where the chosen bins are identical to those used to evaluate the ECE.](pics/calibration.png){#fig:reliability-diagram width="\\textwidth"} Conclusion and Future Work ========================== `\setlength{\intextsep}{-5pt}`{=latex} `\setlength{\columnsep}{10pt}`{=latex} ```{=latex} \begin{wrapfigure}[14]{r}{0.45\textwidth} \includegraphics[width=0.43\textwidth, clip=True, trim=0 0.3cm 0 0]{pics/131-d-model-size-acc-comparison.pdf} \caption{Accuracy vs Model Size.} \label{fig:sota-fine-tuning-model-size-acc} \end{wrapfigure} ``` In this paper, we presented a technical report on Mantis, an [open-source](https://github.com/vfeofanov/mantis), lightweight foundation model for time series classification. Our experimental results show that Mantis outperforms other foundation models while demonstrating superior calibration. We also introduced an adapter approach that offers a user-friendly solution for practitioners with limited computational resources. However, the results suggest that there is still much work to be done in advancing foundation models. First, we believe that designing foundation models for time series classification is a quite challenging task, as can be seen in Figure `\ref{fig:sota-fine-tuning-model-size-acc}`{=latex}, where there is no clear correlation between model size and overall performance. Second, our results reveal a significant performance gap between the zero-shot feature extraction and full fine-tuning regimes, which indicates that there is still a big room for improvement. Finally, we believe that improving the interpretability of the model output is also an important direction for future work. Specifically, ensuring strong calibration properties of the foundation model could enable various applications including uncertainty quantification and unsupervised performance prediction. In our earlier work [@wen2024measuring], we also demonstrated that the hidden space itself can be performance-predictive when using contrastive pre-training. Thus, exploring Mantis from this perspective could be a promising direction for future work. Acknowledgements {#acknowledgements .unnumbered} ================ We would like to thank all the members of Paris Noah's Ark Lab including Youssef Attia El Hili, Ambroise Odonnat, Abdelhakim Benechehab for their constructive comments that helped improve the manuscript. ```{=latex} \bibliographystyle{apalike} ``` ```{=latex} \appendix ``` Experimental Setup {#sec:appendix-exp-setup} ================== Datasets and Benchmarks ----------------------- Below we give more details how datasets were chosen for each benchmark. - *UCR:* For some datasets, the sequences are very long, so fine-tuning GPT4TS does not fit the memory. These are the following 15 datasets: EOGHorizontalSignal, EOGVerticalSignal, EthanolLevel, HandOutlines, InlineSkate, MixedShapesRegularTrain, PLAID, PigAirwayPressure, PigArtPressure, PigCVP, SemgHandGenderCh2, SemgHandMovementCh2, SemgHandSubjectCh2, StarLightCurves, and UWaveGestureLibraryAll. - *UEA:* We have excluded AtrialFibrillation and StandWalkJump datasets due to their very small test size and PenDigits due to its very small sequence length. For InsectWingbeat dataset, we subsampled 1000 examples from the original training set (which contains 30,000 examples) and 1000 from the original test set (of 20,000 examples) to reduce computational overhead while maintaining sufficient variety in the data for robust model evaluation. The following 7 datasets have a very large number of channels, creating issues with full fine-tuning: DuckDuckGeese, FaceDetection, FingerMovements, Heartbeat, InsectWingbeat, MotorImagery, and PEMS-SF. In addition, GPT4TS failed to handle long sequences of the following 4 datasets: EigenWorms, EthanolConcentration, SelfRegulationSCP1, and SelfRegulationSCP2. - *Other datasets:* EMOPain has a lot of channels making the full fine-tuning costly, while MotionSenseHAR has too long sequences for GPT4TS. To sum up, for Section `\ref{sec:sota-zero-shot}`{=latex} and the first experiment of Section `\ref{sec:ablation-study}`{=latex}, we use all 159 datasets without exceptions; for Section `\ref{sec:sota-fine-tuning}`{=latex} and Section `\ref{sec:exp-calibration}`{=latex}, we exclude all long and high-dimensional datasets, i.e., 15 datasets from UCR, 7+4 datasets from UEA, 2 datasets from the others; for the second experiment of Section `\ref{sec:ablation-study}`{=latex}, we exclude only high-dimensional datasets, i.e., 7 datasets from UEA + EMOPain dataset; finally, for Section `\ref{sec:exp-adapters}`{=latex}, we use all 27 UEA datasets under consideration. Implementation Details ---------------------- - *MOMENT:* We use the MOMENT-large model (`d_model`=1024), which pre-trained weights can be found on the corresponding [HuggingFace](https://huggingface.co/AutonLab/MOMENT-1-large) repository. To handle the multi-channel setup, we process every channel independently and concatenate all the embeddings before passing them to the classification head. In the paper, they have considered datasets with a sequence length $\leq 512$ and use zero-padding to fix the input size to 512. At the same time, we have also tried to interpolate sequences to 512 instead, and it did not affect the performance of MOMENT. Thus, we have decided to stick to the latter option as it allows us to evaluate MOMENT for any sequence length. - *GPT4TS:* We use the [code](https://github.com/DAMO-DI-ML/NeurIPS2023-One-Fits-All) provided by the authors. As a backbone, the model uses 6 layers from the pre-trained GPT2, which checkpoint can be found on [HuggingFace](https://huggingface.co/openai-community/gpt2). Channels are processed simultaneously as a part of their tokenizer. Stride and patch size are fixed to 8. - *NuTime.* We use the pre-trained weights provided by the authors in their [GitHub](https://github.com/chenguolin/NuTime/blob/main/ckpt/checkpoint_bias9.pth) repository, while fixing the hyperparameters of the architecture according to [this](https://github.com/chenguolin/NuTime/blob/main/configs/demo_ft_epilepsy.json) configuration file. In contrast to the original implementation, we do not use their adapter (described in Section 3.4 of their paper) but process all channels independently as for Mantis and MOMENT. This allows us to use NuTime in the zero-shot feature extraction setting as their adapter has to be fine-tuned. - *UniTS.* We use the implementation and the checkpoint provided the authors at their [GitHub](https://github.com/mims-harvard/UniTS) repository. Similarly to Mantis, MOMENT and NuTime, channels are processed independently. Stride and patch size are set to 16. Dimension of the model is 64. Complete Results ================ Below we provide full tables with experimental results. ```{=latex} \newcommand{\scalefactor}{0.67} ``` ```{=latex} \newcommand{\scalefactorzeroshot}{0.605} ``` `\scalebox{\scalefactorzeroshot}{ \centering \begin{tabular}{l|lll} \toprule & NuTime & MOMENT & Mantis \\ \midrule ACSF1 & 0.7367$_{\pm0.0351}$ & \textbf{0.75}$_{\pm0.0173}$ & 0.6133$_{\pm0.0208}$ \\ Adiac & 0.7255$_{\pm0.0115}$ & \textbf{0.7886}$_{\pm0.0074}$ & 0.7332$_{\pm0.0039}$ \\ AllGestureWiimoteX & 0.661$_{\pm0.003}$ & 0.6105$_{\pm0.0136}$ & \textbf{0.6705}$_{\pm0.0044}$ \\ AllGestureWiimoteY & 0.6362$_{\pm0.0079}$ & 0.6576$_{\pm0.0079}$ & \textbf{0.6671}$_{\pm0.0057}$ \\ AllGestureWiimoteZ & 0.6024$_{\pm0.0179}$ & 0.5767$_{\pm0.0103}$ & \textbf{0.6695}$_{\pm0.0033}$ \\ ArrowHead & 0.76$_{\pm0.0198}$ & \textbf{0.8076}$_{\pm0.0144}$ & 0.7105$_{\pm0.0175}$ \\ BME & 0.8444$_{\pm0.0168}$ & \textbf{0.9756}$_{\pm0.0139}$ & 0.9311$_{\pm0.0038}$ \\ Beef & 0.6556$_{\pm0.077}$ & \textbf{0.7444}$_{\pm0.0509}$ & 0.6556$_{\pm0.0509}$ \\ BeetleFly & 0.8667$_{\pm0.0289}$ & \textbf{0.95}$_{\pm0.0}$ & 0.85$_{\pm0.05}$ \\ BirdChicken & 0.9667$_{\pm0.0289}$ & 0.85$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ CBF & 0.9744$_{\pm0.0011}$ & 0.9411$_{\pm0.0078}$ & \textbf{0.993}$_{\pm0.0013}$ \\ Car & 0.75$_{\pm0.0}$ & \textbf{0.7944}$_{\pm0.0255}$ & 0.7722$_{\pm0.0419}$ \\ Chinatown & 0.932$_{\pm0.0017}$ & \textbf{0.9806}$_{\pm0.0034}$ & 0.8737$_{\pm0.0168}$ \\ ChlorineConcentration & 0.6675$_{\pm0.0035}$ & \textbf{0.6901}$_{\pm0.0042}$ & 0.6806$_{\pm0.0019}$ \\ CinCECGTorso & \textbf{0.737}$_{\pm0.0152}$ & 0.6998$_{\pm0.0118}$ & 0.6611$_{\pm0.0036}$ \\ Coffee & 0.9167$_{\pm0.0206}$ & 0.8929$_{\pm0.0}$ & \textbf{0.9524}$_{\pm0.0206}$ \\ Computers & \textbf{0.78}$_{\pm0.004}$ & 0.6173$_{\pm0.0162}$ & 0.7373$_{\pm0.0092}$ \\ CricketX & 0.6701$_{\pm0.0146}$ & 0.6795$_{\pm0.0051}$ & \textbf{0.7368}$_{\pm0.0171}$ \\ CricketY & 0.6556$_{\pm0.0053}$ & 0.6897$_{\pm0.0077}$ & \textbf{0.7504}$_{\pm0.0065}$ \\ CricketZ & 0.6863$_{\pm0.0171}$ & 0.7128$_{\pm0.0044}$ & \textbf{0.7906}$_{\pm0.0039}$ \\ Crop & 0.6683$_{\pm0.0026}$ & \textbf{0.7035}$_{\pm0.0026}$ & 0.6756$_{\pm0.0018}$ \\ DiatomSizeReduction & 0.8322$_{\pm0.0115}$ & \textbf{0.8867}$_{\pm0.0019}$ & 0.8845$_{\pm0.0019}$ \\ DistalPhalanxOutlineAgeGroup & 0.7362$_{\pm0.011}$ & 0.7506$_{\pm0.011}$ & \textbf{0.789}$_{\pm0.015}$ \\ DistalPhalanxOutlineCorrect & 0.7742$_{\pm0.0055}$ & \textbf{0.7886}$_{\pm0.0055}$ & 0.75$_{\pm0.0126}$ \\ DistalPhalanxTW & 0.6763$_{\pm0.0}$ & 0.6571$_{\pm0.011}$ & \textbf{0.6859}$_{\pm0.011}$ \\ DodgerLoopDay & 0.5167$_{\pm0.0361}$ & 0.4542$_{\pm0.0144}$ & \textbf{0.55}$_{\pm0.0217}$ \\ DodgerLoopGame & 0.756$_{\pm0.0084}$ & \textbf{0.8116}$_{\pm0.0126}$ & 0.7585$_{\pm0.0221}$ \\ DodgerLoopWeekend & 0.9565$_{\pm0.0072}$ & \textbf{0.9614}$_{\pm0.0111}$ & 0.9517$_{\pm0.0084}$ \\ ECG200 & 0.8133$_{\pm0.0153}$ & \textbf{0.8967}$_{\pm0.0153}$ & 0.82$_{\pm0.01}$ \\ ECG5000 & 0.9313$_{\pm0.0003}$ & \textbf{0.9384}$_{\pm0.0008}$ & 0.9211$_{\pm0.001}$ \\ ECGFiveDays & 0.7801$_{\pm0.017}$ & 0.8564$_{\pm0.0223}$ & \textbf{0.909}$_{\pm0.0218}$ \\ EOGHorizontalSignal & 0.4346$_{\pm0.0032}$ & 0.5571$_{\pm0.0112}$ & \textbf{0.5875}$_{\pm0.0089}$ \\ EOGVerticalSignal & 0.2716$_{\pm0.008}$ & 0.4595$_{\pm0.0097}$ & \textbf{0.4751}$_{\pm0.0}$ \\ Earthquakes & 0.7458$_{\pm0.0042}$ & 0.7458$_{\pm0.0042}$ & \textbf{0.7482}$_{\pm0.0}$ \\ ElectricDevices & 0.7046$_{\pm0.0014}$ & 0.7142$_{\pm0.001}$ & \textbf{0.7226}$_{\pm0.0026}$ \\ EthanolLevel & 0.3407$_{\pm0.0101}$ & \textbf{0.4227}$_{\pm0.0058}$ & 0.2993$_{\pm0.011}$ \\ FaceAll & 0.6363$_{\pm0.0053}$ & 0.7398$_{\pm0.0074}$ & \textbf{0.7815}$_{\pm0.0074}$ \\ FaceFour & 0.7841$_{\pm0.0521}$ & 0.7765$_{\pm0.0174}$ & \textbf{0.9508}$_{\pm0.0066}$ \\ FacesUCR & 0.7141$_{\pm0.003}$ & 0.7951$_{\pm0.0034}$ & \textbf{0.8354}$_{\pm0.0054}$ \\ FiftyWords & 0.5949$_{\pm0.0125}$ & \textbf{0.6777}$_{\pm0.0111}$ & 0.6462$_{\pm0.0096}$ \\ Fish & 0.9162$_{\pm0.0033}$ & 0.8705$_{\pm0.0201}$ & \textbf{0.9333}$_{\pm0.0066}$ \\ FordA & 0.8932$_{\pm0.002}$ & \textbf{0.9015}$_{\pm0.0008}$ & 0.8581$_{\pm0.0048}$ \\ FordB & 0.7642$_{\pm0.0119}$ & \textbf{0.765}$_{\pm0.0019}$ & 0.7305$_{\pm0.0031}$ \\ FreezerRegularTrain & \textbf{0.9738}$_{\pm0.0013}$ & 0.8994$_{\pm0.0012}$ & 0.9374$_{\pm0.0043}$ \\ FreezerSmallTrain & \textbf{0.9549}$_{\pm0.0059}$ & 0.7758$_{\pm0.0067}$ & 0.7942$_{\pm0.0059}$ \\ Fungi & 0.7043$_{\pm0.0093}$ & \textbf{0.9964}$_{\pm0.0062}$ & 0.8262$_{\pm0.0164}$ \\ GestureMidAirD1 & \textbf{0.6744}$_{\pm0.0379}$ & \textbf{0.6744}$_{\pm0.0044}$ & 0.659$_{\pm0.0044}$ \\ GestureMidAirD2 & 0.5692$_{\pm0.0}$ & 0.5744$_{\pm0.016}$ & \textbf{0.6154}$_{\pm0.0077}$ \\ GestureMidAirD3 & \textbf{0.3974}$_{\pm0.0247}$ & 0.359$_{\pm0.0118}$ & 0.3282$_{\pm0.016}$ \\ GesturePebbleZ1 & 0.8915$_{\pm0.0067}$ & 0.8469$_{\pm0.0067}$ & \textbf{0.9283}$_{\pm0.0034}$ \\ GesturePebbleZ2 & 0.8165$_{\pm0.011}$ & 0.8376$_{\pm0.0073}$ & \textbf{0.9219}$_{\pm0.0256}$ \\ GunPoint & 0.9444$_{\pm0.0038}$ & \textbf{1.0}$_{\pm0.0}$ & 0.98$_{\pm0.0067}$ \\ GunPointAgeSpan & 0.9684$_{\pm0.0032}$ & 0.962$_{\pm0.0032}$ & \textbf{0.9905}$_{\pm0.0}$ \\ GunPointMaleVersusFemale & 0.962$_{\pm0.0032}$ & 0.9884$_{\pm0.0018}$ & \textbf{0.9958}$_{\pm0.0018}$ \\ GunPointOldVersusYoung & \textbf{1.0}$_{\pm0.0}$ & 0.9577$_{\pm0.008}$ & 0.9968$_{\pm0.0}$ \\ Ham & 0.7206$_{\pm0.022}$ & \textbf{0.7714}$_{\pm0.0}$ & 0.673$_{\pm0.0145}$ \\ HandOutlines & 0.9045$_{\pm0.0062}$ & 0.909$_{\pm0.0056}$ & \textbf{0.9162}$_{\pm0.0072}$ \\ Haptics & 0.4481$_{\pm0.0056}$ & \textbf{0.5152}$_{\pm0.0068}$ & 0.4968$_{\pm0.0032}$ \\ Herring & 0.599$_{\pm0.0325}$ & 0.6146$_{\pm0.009}$ & \textbf{0.6667}$_{\pm0.0239}$ \\ HouseTwenty & 0.8655$_{\pm0.0084}$ & 0.9356$_{\pm0.0097}$ & \textbf{0.9412}$_{\pm0.0}$ \\ InlineSkate & 0.357$_{\pm0.0105}$ & 0.3176$_{\pm0.001}$ & \textbf{0.363}$_{\pm0.0136}$ \\ InsectEPGRegularTrain & \textbf{1.0}$_{\pm0.0}$ & 0.9183$_{\pm0.0061}$ & \textbf{1.0}$_{\pm0.0}$ \\ InsectEPGSmallTrain & \textbf{1.0}$_{\pm0.0}$ & 0.8501$_{\pm0.0061}$ & \textbf{1.0}$_{\pm0.0}$ \\ InsectWingbeatSound & 0.5066$_{\pm0.0066}$ & \textbf{0.6158}$_{\pm0.0037}$ & 0.519$_{\pm0.0044}$ \\ \midrule Blink & 0.6926$_{\pm0.0156}$ & 0.9911$_{\pm0.0022}$ & \textbf{0.9993}$_{\pm0.0013}$ \\ EMOPain & \textbf{0.8836}$_{\pm0.0016}$ & 0.8432$_{\pm0.0091}$ & 0.8385$_{\pm0.0016}$ \\ MotionSenseHAR & 0.9887$_{\pm0.0}$ & 0.9673$_{\pm0.0022}$ & \textbf{0.9975}$_{\pm0.0022}$\\ SharePriceIncrease & \textbf{0.6932}$_{\pm0.0006}$ & 0.6832$_{\pm0.0082}$ & 0.6836$_{\pm0.0049}$ \\ \midrule ArticularyWordRecognition & \textbf{0.9933}$_{\pm0.0}$ & 0.98$_{\pm0.0}$ & 0.993$_{\pm0.003}$ \\ BasicMotions & \textbf{1.0}$_{\pm0.0}$ & 0.992$_{\pm0.014}$ & \textbf{1.0}$_{\pm0.0}$ \\ CharacterTrajectories & 0.9649$_{\pm0.0038}$ & \textbf{0.97}$_{\pm0.002}$ & 0.94$_{\pm0.001}$ \\ Cricket & 0.9907$_{\pm0.008}$ & 0.977$_{\pm0.008}$ & \textbf{1.0}$_{\pm0.0}$ \\ DuckDuckGeese & 0.4267$_{\pm0.0503}$ & \textbf{0.433}$_{\pm0.031}$ & 0.407$_{\pm0.031}$ \\ ERing & \textbf{0.9716}$_{\pm0.0057}$ & 0.969$_{\pm0.008}$ & 0.941$_{\pm0.01}$ \\ EigenWorms & \textbf{0.7786}$_{\pm0.0202}$ & 0.735$_{\pm0.016}$ & 0.753$_{\pm0.016}$ \\ Epilepsy & \textbf{1.0}$_{\pm0.0}$ & 0.988$_{\pm0.004}$ & 0.995$_{\pm0.004}$ \\ EthanolConcentration & \textbf{0.3942}$_{\pm0.0158}$ & 0.278$_{\pm0.008}$ & 0.27$_{\pm0.01}$ \\ FaceDetection & \textbf{0.5581}$_{\pm0.0077}$ & 0.544$_{\pm0.011}$ & 0.526$_{\pm0.002}$ \\ FingerMovements & 0.5133$_{\pm0.0153}$ & 0.497$_{\pm0.023}$ & \textbf{0.54}$_{\pm0.035}$ \\ HandMovementDirection & 0.3108$_{\pm0.0234}$ & \textbf{0.315}$_{\pm0.008}$ & 0.212$_{\pm0.041}$ \\ \bottomrule \end{tabular} }`{=latex} ```{=latex} \hfill ``` `\scalebox{\scalefactorzeroshot}{ \centering \begin{tabular}{l|lll} \toprule & NuTime & MOMENT & Mantis \\ \midrule ItalyPowerDemand & 0.8698$_{\pm0.007}$ & \textbf{0.9498}$_{\pm0.0006}$ & 0.9077$_{\pm0.0035}$ \\ LargeKitchenAppliances & 0.7227$_{\pm0.0027}$ & 0.7333$_{\pm0.0071}$ & \textbf{0.7804}$_{\pm0.0041}$ \\ Lightning2 & 0.6885$_{\pm0.0164}$ & 0.7377$_{\pm0.0328}$ & \textbf{0.8033}$_{\pm0.0}$ \\ Lightning7 & 0.6895$_{\pm0.0158}$ & 0.6758$_{\pm0.0209}$ & \textbf{0.7763}$_{\pm0.0285}$ \\ Mallat & 0.8304$_{\pm0.0025}$ & 0.859$_{\pm0.0111}$ & \textbf{0.8903}$_{\pm0.0137}$ \\ Meat & 0.8889$_{\pm0.0192}$ & \textbf{0.9444}$_{\pm0.0096}$ & 0.9389$_{\pm0.0096}$ \\ MedicalImages & 0.7044$_{\pm0.0027}$ & 0.6939$_{\pm0.0062}$ & \textbf{0.7079}$_{\pm0.0035}$ \\ MelbournePedestrian & \textbf{0.912}$_{\pm0.0045}$ & 0.8662$_{\pm0.0047}$ & 0.9016$_{\pm0.0031}$ \\ MiddlePhalanxOutlineAgeGroup & \textbf{0.6169}$_{\pm0.0065}$ & 0.5779$_{\pm0.0065}$ & 0.5801$_{\pm0.0099}$ \\ MiddlePhalanxOutlineCorrect & 0.7858$_{\pm0.0099}$ & \textbf{0.8625}$_{\pm0.006}$ & 0.8099$_{\pm0.0099}$ \\ MiddlePhalanxTW & 0.5238$_{\pm0.0037}$ & \textbf{0.5952}$_{\pm0.0037}$ & 0.5368$_{\pm0.0099}$ \\ MixedShapesRegularTrain & 0.9392$_{\pm0.0013}$ & 0.9124$_{\pm0.0023}$ & \textbf{0.943}$_{\pm0.0044}$ \\ MixedShapesSmallTrain & \textbf{0.9061}$_{\pm0.0029}$ & 0.8389$_{\pm0.0041}$ & 0.8961$_{\pm0.0004}$ \\ MoteStrain & \textbf{0.9462}$_{\pm0.0032}$ & 0.8914$_{\pm0.0083}$ & 0.9137$_{\pm0.0136}$ \\ NonInvasiveFetalECGThorax1 & 0.7696$_{\pm0.0049}$ & \textbf{0.8887}$_{\pm0.0035}$ & 0.6222$_{\pm0.0037}$ \\ NonInvasiveFetalECGThorax2 & 0.811$_{\pm0.0016}$ & \textbf{0.9138}$_{\pm0.0006}$ & 0.6872$_{\pm0.0025}$ \\ OSULeaf & 0.7975$_{\pm0.0072}$ & 0.7355$_{\pm0.0041}$ & \textbf{0.8747}$_{\pm0.0104}$ \\ OliveOil & 0.7111$_{\pm0.0192}$ & 0.9$_{\pm0.0}$ & \textbf{0.9333}$_{\pm0.0}$ \\ PLAID & 0.7933$_{\pm0.0113}$ & 0.7312$_{\pm0.0088}$ & \textbf{0.8181}$_{\pm0.0047}$ \\ PhalangesOutlinesCorrect & 0.7743$_{\pm0.0041}$ & \textbf{0.8248}$_{\pm0.0007}$ & 0.7786$_{\pm0.0042}$ \\ Phoneme & 0.2802$_{\pm0.0044}$ & 0.2751$_{\pm0.0062}$ & \textbf{0.323}$_{\pm0.0034}$ \\ PickupGestureWiimoteZ & 0.6933$_{\pm0.0808}$ & 0.68$_{\pm0.06}$ & \textbf{0.74}$_{\pm0.02}$ \\ PigAirwayPressure & 0.3782$_{\pm0.01}$ & 0.1186$_{\pm0.0028}$ & \textbf{0.484}$_{\pm0.0147}$ \\ PigArtPressure & \textbf{0.9391}$_{\pm0.0028}$ & 0.6106$_{\pm0.0048}$ & 0.9103$_{\pm0.0028}$ \\ PigCVP & \textbf{0.8381}$_{\pm0.0242}$ & 0.609$_{\pm0.0373}$ & 0.7837$_{\pm0.0127}$ \\ Plane & 0.9937$_{\pm0.0055}$ & 0.9841$_{\pm0.0145}$ & \textbf{1.0}$_{\pm0.0}$ \\ PowerCons & 0.9352$_{\pm0.0032}$ & \textbf{0.9463}$_{\pm0.0032}$ & 0.9093$_{\pm0.0032}$ \\ ProximalPhalanxOutlineAgeGroup & \textbf{0.8537}$_{\pm0.0049}$ & 0.839$_{\pm0.0049}$ & \textbf{0.8537}$_{\pm0.0049}$ \\ ProximalPhalanxOutlineCorrect & 0.8385$_{\pm0.0034}$ & \textbf{0.8751}$_{\pm0.0052}$ & 0.8202$_{\pm0.0086}$ \\ ProximalPhalanxTW & 0.8065$_{\pm0.0056}$ & \textbf{0.8114}$_{\pm0.0028}$ & 0.7691$_{\pm0.0028}$ \\ RefrigerationDevices & \textbf{0.5564}$_{\pm0.0041}$ & 0.536$_{\pm0.0046}$ & 0.504$_{\pm0.0122}$ \\ Rock & 0.6067$_{\pm0.0306}$ & \textbf{0.84}$_{\pm0.02}$ & 0.7133$_{\pm0.0115}$ \\ ScreenType & \textbf{0.5324}$_{\pm0.0077}$ & 0.4436$_{\pm0.0178}$ & 0.4649$_{\pm0.0134}$ \\ SemgHandGenderCh2 & 0.8928$_{\pm0.0042}$ & 0.8028$_{\pm0.0079}$ & \textbf{0.9189}$_{\pm0.0086}$ \\ SemgHandMovementCh2 & 0.72$_{\pm0.0038}$ & 0.5237$_{\pm0.0013}$ & \textbf{0.797}$_{\pm0.0056}$ \\ SemgHandSubjectCh2 & 0.7741$_{\pm0.0134}$ & 0.6815$_{\pm0.0026}$ & \textbf{0.8622}$_{\pm0.0135}$ \\ ShakeGestureWiimoteZ & \textbf{0.9267}$_{\pm0.0115}$ & 0.8533$_{\pm0.0115}$ & 0.8867$_{\pm0.0115}$ \\ ShapeletSim & 0.8833$_{\pm0.0111}$ & \textbf{0.9648}$_{\pm0.0064}$ & 0.9278$_{\pm0.0111}$ \\ ShapesAll & 0.8194$_{\pm0.0025}$ & \textbf{0.8239}$_{\pm0.0092}$ & 0.8194$_{\pm0.0054}$ \\ SmallKitchenAppliances & 0.8027$_{\pm0.0071}$ & 0.72$_{\pm0.0141}$ & \textbf{0.8089}$_{\pm0.0081}$ \\ SmoothSubspace & 0.8933$_{\pm0.0067}$ & \textbf{0.92}$_{\pm0.0133}$ & 0.9067$_{\pm0.0115}$ \\ SonyAIBORobotSurface1 & 0.7987$_{\pm0.0044}$ & \textbf{0.8087}$_{\pm0.006}$ & 0.787$_{\pm0.0294}$ \\ SonyAIBORobotSurface2 & 0.8297$_{\pm0.0034}$ & 0.8353$_{\pm0.0106}$ & \textbf{0.8552}$_{\pm0.0168}$ \\ StarLightCurves & \textbf{0.9792}$_{\pm0.0002}$ & 0.9734$_{\pm0.0004}$ & 0.9759$_{\pm0.0005}$ \\ Strawberry & 0.9378$_{\pm0.0072}$ & \textbf{0.9604}$_{\pm0.0016}$ & 0.9514$_{\pm0.0027}$ \\ SwedishLeaf & 0.9205$_{\pm0.0065}$ & 0.9205$_{\pm0.0051}$ & \textbf{0.9275}$_{\pm0.0009}$ \\ Symbols & 0.9407$_{\pm0.0126}$ & 0.938$_{\pm0.0126}$ & \textbf{0.9698}$_{\pm0.0056}$ \\ SyntheticControl & 0.9667$_{\pm0.0033}$ & 0.9433$_{\pm0.0}$ & \textbf{0.9767}$_{\pm0.0067}$ \\ ToeSegmentation1 & 0.8596$_{\pm0.0044}$ & 0.924$_{\pm0.0127}$ & \textbf{0.9635}$_{\pm0.0067}$ \\ ToeSegmentation2 & 0.7256$_{\pm0.016}$ & 0.8462$_{\pm0.0077}$ & \textbf{0.9282}$_{\pm0.0044}$ \\ Trace & \textbf{1.0}$_{\pm0.0}$ & 0.99$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ TwoLeadECG & 0.8712$_{\pm0.0303}$ & 0.9649$_{\pm0.0116}$ & \textbf{0.9962}$_{\pm0.0005}$ \\ TwoPatterns & 0.8414$_{\pm0.0041}$ & \textbf{0.8919}$_{\pm0.0079}$ & 0.8802$_{\pm0.0079}$ \\ UMD & 0.9329$_{\pm0.0223}$ & 0.9699$_{\pm0.004}$ & \textbf{0.9722}$_{\pm0.0069}$ \\ UWaveGestureLibraryAll & 0.8773$_{\pm0.0028}$ & \textbf{0.8975}$_{\pm0.0021}$ & 0.8458$_{\pm0.0057}$ \\ UWaveGestureLibraryX & \textbf{0.8053}$_{\pm0.0007}$ & 0.7829$_{\pm0.0071}$ & 0.7696$_{\pm0.0028}$ \\ UWaveGestureLibraryY & \textbf{0.7338}$_{\pm0.0032}$ & 0.7022$_{\pm0.0026}$ & 0.6874$_{\pm0.0022}$ \\ UWaveGestureLibraryZ & \textbf{0.7427}$_{\pm0.0036}$ & 0.73$_{\pm0.002}$ & 0.7324$_{\pm0.0036}$ \\ Wafer & \textbf{0.9933}$_{\pm0.0006}$ & 0.9867$_{\pm0.0003}$ & 0.9903$_{\pm0.0003}$ \\ Wine & 0.7469$_{\pm0.0107}$ & \textbf{0.8457}$_{\pm0.0283}$ & 0.7901$_{\pm0.0107}$ \\ WordSynonyms & 0.5199$_{\pm0.0127}$ & \textbf{0.6003}$_{\pm0.0078}$ & 0.5502$_{\pm0.0081}$ \\ Worms & \textbf{0.7403}$_{\pm0.0}$ & 0.7056$_{\pm0.015}$ & 0.6537$_{\pm0.027}$ \\ WormsTwoClass & 0.7835$_{\pm0.0075}$ & 0.7706$_{\pm0.0075}$ & \textbf{0.8139}$_{\pm0.0198}$ \\ Yoga & 0.8219$_{\pm0.0015}$ & \textbf{0.8264}$_{\pm0.003}$ & 0.815$_{\pm0.0012}$ \\ \midrule Handwriting & 0.2035$_{\pm0.0082}$ & 0.236$_{\pm0.005}$ & \textbf{0.339}$_{\pm0.01}$ \\ Heartbeat & 0.7756$_{\pm0.0}$ & 0.727$_{\pm0.005}$ & \textbf{0.798}$_{\pm0.017}$ \\ InsectWingbeatSubset & \textbf{0.611}$_{\pm0.0066}$ & 0.243$_{\pm0.011}$ & 0.607$_{\pm0.007}$ \\ JapaneseVowels & 0.9405$_{\pm0.0072}$ & 0.88$_{\pm0.009}$ & \textbf{0.963}$_{\pm0.006}$ \\ LSST & 0.5668$_{\pm0.0068}$ & \textbf{0.621}$_{\pm0.001}$ & 0.605$_{\pm0.002}$ \\ Libras & 0.8852$_{\pm0.0064}$ & 0.85$_{\pm0.02}$ & \textbf{0.898}$_{\pm0.008}$ \\ MotorImagery & 0.5367$_{\pm0.0153}$ & 0.547$_{\pm0.055}$ & \textbf{0.55}$_{\pm0.036}$ \\ NATOPS & 0.8426$_{\pm0.014}$ & 0.822$_{\pm0.024}$ & \textbf{0.907}$_{\pm0.013}$ \\ PEMS-SF & 0.9884$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.985$_{\pm0.013}$ \\ PhonemeSpectra & 0.2588$_{\pm0.0065}$ & 0.215$_{\pm0.001}$ & \textbf{0.273}$_{\pm0.008}$ \\ RacketSports & 0.9101$_{\pm0.01}$ & 0.82$_{\pm0.01}$ & \textbf{0.923}$_{\pm0.004}$ \\ SelfRegulationSCP1 & 0.7702$_{\pm0.012}$ & 0.754$_{\pm0.012}$ & \textbf{0.804}$_{\pm0.011}$ \\ SelfRegulationSCP2 & \textbf{0.5}$_{\pm0.0242}$ & 0.496$_{\pm0.02}$ & 0.476$_{\pm0.045}$ \\ SpokenArabicDigits & 0.8968$_{\pm0.0032}$ & \textbf{0.935}$_{\pm0.002}$ & 0.839$_{\pm0.005}$ \\ UWaveGestureLibrary & \textbf{0.8792}$_{\pm0.0018}$ & 0.873$_{\pm0.011}$ & 0.814$_{\pm0.01}$ \\ \midrule \textit{\textbf{Best Count}} & 38 & 53 & \textbf{74} \\ \bottomrule \end{tabular} }`{=latex} ```{=latex} \newcommand{\scalefactorsotaft}{0.46} ``` `\scalebox{\scalefactorsotaft}{ \begin{tabular}{l|lllll} \toprule & UniTS & GPT4TS & NuTime & MOMENT & Mantis \\ \midrule ACSF1 & 0.6933$_{\pm0.0351}$ & 0.49$_{\pm0.0265}$ & 0.6733$_{\pm0.0321}$ & 0.6033$_{\pm0.0379}$ & \textbf{0.7433}$_{\pm0.0115}$ \\ Adiac & 0.5772$_{\pm0.0074}$ & 0.3572$_{\pm0.0059}$ & 0.7349$_{\pm0.0141}$ & 0.5823$_{\pm0.0296}$ & \textbf{0.7766}$_{\pm0.003}$ \\ AllGestureWiimoteX & 0.469$_{\pm0.0224}$ & 0.4895$_{\pm0.0105}$ & 0.6386$_{\pm0.0038}$ & 0.7$_{\pm0.0103}$ & \textbf{0.7619}$_{\pm0.005}$ \\ AllGestureWiimoteY & 0.5338$_{\pm0.0242}$ & 0.5062$_{\pm0.0116}$ & 0.7152$_{\pm0.0103}$ & 0.7081$_{\pm0.0343}$ & \textbf{0.7948}$_{\pm0.0097}$ \\ AllGestureWiimoteZ & 0.4595$_{\pm0.0159}$ & 0.4719$_{\pm0.0119}$ & 0.6419$_{\pm0.0058}$ & 0.6976$_{\pm0.0218}$ & \textbf{0.73}$_{\pm0.01}$ \\ ArrowHead & 0.6895$_{\pm0.0087}$ & 0.7638$_{\pm0.0033}$ & \textbf{0.8305}$_{\pm0.0033}$ & 0.779$_{\pm0.0682}$ & 0.821$_{\pm0.0389}$ \\ BME & 0.8867$_{\pm0.1048}$ & 0.9444$_{\pm0.0102}$ & \textbf{1.0}$_{\pm0.0}$ & 0.98$_{\pm0.0231}$ & 0.9956$_{\pm0.0077}$ \\ Beef & 0.6667$_{\pm0.0333}$ & 0.8$_{\pm0.0577}$ & \textbf{0.8222}$_{\pm0.0192}$ & 0.8$_{\pm0.0577}$ & 0.7$_{\pm0.0333}$ \\ BeetleFly & 0.7167$_{\pm0.0289}$ & 0.8167$_{\pm0.0289}$ & \textbf{0.8833}$_{\pm0.0764}$ & 0.8$_{\pm0.15}$ & \textbf{0.8833}$_{\pm0.0764}$ \\ BirdChicken & 0.6167$_{\pm0.1155}$ & 0.6167$_{\pm0.0289}$ & 0.85$_{\pm0.05}$ & 0.7667$_{\pm0.0289}$ & \textbf{0.9}$_{\pm0.05}$ \\ CBF & 0.8507$_{\pm0.0192}$ & 0.8596$_{\pm0.0083}$ & 0.9652$_{\pm0.0046}$ & 0.9767$_{\pm0.0172}$ & \textbf{0.9848}$_{\pm0.0074}$ \\ Car & 0.6278$_{\pm0.0509}$ & 0.8$_{\pm0.0167}$ & 0.8444$_{\pm0.0096}$ & \textbf{0.8778}$_{\pm0.0419}$ & 0.8722$_{\pm0.0096}$ \\ Chinatown & \textbf{0.9776}$_{\pm0.0045}$ & 0.9718$_{\pm0.0089}$ & 0.9738$_{\pm0.0}$ & 0.9708$_{\pm0.0127}$ & 0.9718$_{\pm0.0017}$ \\ ChlorineConcentration & 0.6424$_{\pm0.0203}$ & 0.5733$_{\pm0.0091}$ & 0.6718$_{\pm0.008}$ & 0.6289$_{\pm0.0178}$ & \textbf{0.6971}$_{\pm0.0112}$ \\ CinCECGTorso & 0.585$_{\pm0.0324}$ & 0.793$_{\pm0.0353}$ & \textbf{0.9302}$_{\pm0.0067}$ & 0.743$_{\pm0.008}$ & 0.7814$_{\pm0.0173}$ \\ Coffee & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.9881$_{\pm0.0206}$ & \textbf{1.0}$_{\pm0.0}$ \\ Computers & 0.6093$_{\pm0.0046}$ & 0.62$_{\pm0.0106}$ & \textbf{0.812}$_{\pm0.0174}$ & 0.6747$_{\pm0.0295}$ & 0.7813$_{\pm0.0162}$ \\ CricketX & 0.5145$_{\pm0.0074}$ & 0.5231$_{\pm0.0143}$ & 0.7453$_{\pm0.0039}$ & 0.7915$_{\pm0.0192}$ & \textbf{0.7966}$_{\pm0.009}$ \\ CricketY & 0.541$_{\pm0.0194}$ & 0.5419$_{\pm0.009}$ & 0.7479$_{\pm0.0104}$ & 0.7487$_{\pm0.0271}$ & \textbf{0.806}$_{\pm0.0141}$ \\ CricketZ & 0.5521$_{\pm0.0292}$ & 0.5496$_{\pm0.0039}$ & 0.7701$_{\pm0.0246}$ & \textbf{0.8068}$_{\pm0.0218}$ & 0.8043$_{\pm0.0415}$ \\ Crop & 0.7184$_{\pm0.0039}$ & 0.7311$_{\pm0.0033}$ & \textbf{0.7574}$_{\pm0.0022}$ & 0.717$_{\pm0.0071}$ & 0.7444$_{\pm0.0039}$ \\ DiatomSizeReduction & 0.9118$_{\pm0.0259}$ & 0.9401$_{\pm0.0361}$ & 0.9314$_{\pm0.0425}$ & 0.9325$_{\pm0.0576}$ & \textbf{0.9684}$_{\pm0.0136}$ \\ DistalPhalanxOutlineAgeGroup & 0.7434$_{\pm0.0083}$ & 0.7218$_{\pm0.0042}$ & 0.7386$_{\pm0.015}$ & 0.7362$_{\pm0.0272}$ & \textbf{0.7698}$_{\pm0.0259}$ \\ DistalPhalanxOutlineCorrect & 0.75$_{\pm0.0166}$ & 0.6932$_{\pm0.0084}$ & 0.7705$_{\pm0.0302}$ & 0.7452$_{\pm0.02}$ & \textbf{0.7717}$_{\pm0.0158}$ \\ DistalPhalanxTW & 0.6691$_{\pm0.0125}$ & \textbf{0.6978}$_{\pm0.0125}$ & 0.6978$_{\pm0.0216}$ & 0.6451$_{\pm0.03}$ & 0.6954$_{\pm0.0231}$ \\ DodgerLoopDay & 0.45$_{\pm0.0331}$ & 0.5542$_{\pm0.0564}$ & 0.5458$_{\pm0.0688}$ & 0.5292$_{\pm0.0641}$ & \textbf{0.625}$_{\pm0.025}$ \\ DodgerLoopGame & 0.8309$_{\pm0.0233}$ & 0.8551$_{\pm0.0192}$ & 0.8213$_{\pm0.0649}$ & 0.8599$_{\pm0.0233}$ & \textbf{0.8841}$_{\pm0.0145}$ \\ DodgerLoopWeekend & 0.9638$_{\pm0.0072}$ & \textbf{0.9855}$_{\pm0.0}$ & 0.9783$_{\pm0.0126}$ & 0.9783$_{\pm0.0126}$ & 0.9783$_{\pm0.0}$ \\ ECG200 & 0.8767$_{\pm0.0611}$ & 0.8467$_{\pm0.0551}$ & 0.87$_{\pm0.01}$ & \textbf{0.8867}$_{\pm0.0231}$ & 0.8567$_{\pm0.0058}$ \\ ECG5000 & 0.9314$_{\pm0.0042}$ & \textbf{0.9408}$_{\pm0.0019}$ & 0.9379$_{\pm0.0001}$ & 0.9325$_{\pm0.0003}$ & 0.9335$_{\pm0.0081}$ \\ ECGFiveDays & 0.8459$_{\pm0.054}$ & 0.8784$_{\pm0.076}$ & \textbf{0.9621}$_{\pm0.0084}$ & 0.7944$_{\pm0.0373}$ & 0.9148$_{\pm0.0292}$ \\ Earthquakes & 0.7386$_{\pm0.03}$ & 0.7338$_{\pm0.0072}$ & 0.7482$_{\pm0.0}$ & \textbf{0.7578}$_{\pm0.0166}$ & 0.7482$_{\pm0.0}$ \\ ElectricDevices & 0.6194$_{\pm0.0046}$ & 0.5776$_{\pm0.0173}$ & 0.7112$_{\pm0.0109}$ & 0.6666$_{\pm0.0204}$ & \textbf{0.7454}$_{\pm0.0049}$ \\ FaceAll & 0.7335$_{\pm0.0343}$ & 0.7247$_{\pm0.0042}$ & \textbf{0.8385}$_{\pm0.0098}$ & 0.8012$_{\pm0.008}$ & 0.8308$_{\pm0.0031}$ \\ FaceFour & 0.6515$_{\pm0.0347}$ & 0.8295$_{\pm0.0301}$ & 0.9318$_{\pm0.0227}$ & 0.8371$_{\pm0.0853}$ & \textbf{0.9773}$_{\pm0.0114}$ \\ FacesUCR & 0.7197$_{\pm0.0078}$ & 0.7932$_{\pm0.0047}$ & 0.8846$_{\pm0.0127}$ & 0.835$_{\pm0.0014}$ & \textbf{0.9148}$_{\pm0.0071}$ \\ FiftyWords & 0.6227$_{\pm0.0104}$ & 0.6615$_{\pm0.0058}$ & 0.7875$_{\pm0.0083}$ & 0.7861$_{\pm0.0222}$ & \textbf{0.8139}$_{\pm0.0083}$ \\ Fish & 0.7943$_{\pm0.0396}$ & 0.8133$_{\pm0.0119}$ & 0.9524$_{\pm0.0119}$ & 0.9162$_{\pm0.0201}$ & \textbf{0.9714}$_{\pm0.0099}$ \\ FordA & 0.9202$_{\pm0.0049}$ & 0.7207$_{\pm0.0798}$ & 0.9225$_{\pm0.0095}$ & 0.9202$_{\pm0.0271}$ & \textbf{0.9343}$_{\pm0.0046}$ \\ FordB & 0.7634$_{\pm0.0043}$ & 0.6107$_{\pm0.0567}$ & 0.7922$_{\pm0.0126}$ & \textbf{0.8173}$_{\pm0.0125}$ & 0.7979$_{\pm0.0135}$ \\ FreezerRegularTrain & 0.911$_{\pm0.0171}$ & 0.8884$_{\pm0.0329}$ & \textbf{0.9942}$_{\pm0.0014}$ & 0.9876$_{\pm0.0056}$ & 0.9927$_{\pm0.0041}$ \\ FreezerSmallTrain & 0.6713$_{\pm0.012}$ & 0.6614$_{\pm0.0094}$ & 0.9537$_{\pm0.029}$ & 0.8206$_{\pm0.0695}$ & \textbf{0.9667}$_{\pm0.0098}$ \\ Fungi & 0.6272$_{\pm0.0306}$ & 0.7061$_{\pm0.0124}$ & 0.7294$_{\pm0.0135}$ & 0.69$_{\pm0.035}$ & \textbf{0.7778}$_{\pm0.0031}$ \\ GestureMidAirD1 & 0.5256$_{\pm0.0347}$ & 0.5615$_{\pm0.0353}$ & 0.741$_{\pm0.0222}$ & 0.6487$_{\pm0.0311}$ & \textbf{0.7692}$_{\pm0.0407}$ \\ GestureMidAirD2 & 0.4051$_{\pm0.0355}$ & 0.4846$_{\pm0.0428}$ & 0.6513$_{\pm0.0347}$ & 0.5615$_{\pm0.0077}$ & \textbf{0.6744}$_{\pm0.0118}$ \\ GestureMidAirD3 & 0.2949$_{\pm0.0387}$ & 0.3308$_{\pm0.04}$ & \textbf{0.4359}$_{\pm0.0089}$ & 0.3615$_{\pm0.0277}$ & 0.4077$_{\pm0.0231}$ \\ GesturePebbleZ1 & 0.7074$_{\pm0.032}$ & 0.8217$_{\pm0.0034}$ & 0.8953$_{\pm0.0058}$ & 0.9109$_{\pm0.0089}$ & \textbf{0.9322}$_{\pm0.0089}$ \\ GesturePebbleZ2 & 0.7089$_{\pm0.0127}$ & 0.7553$_{\pm0.0159}$ & 0.8861$_{\pm0.0219}$ & 0.8861$_{\pm0.0063}$ & \textbf{0.9241}$_{\pm0.0219}$ \\ GunPoint & 0.9089$_{\pm0.0252}$ & 0.8733$_{\pm0.0067}$ & 0.9933$_{\pm0.0}$ & 0.9867$_{\pm0.0115}$ & \textbf{1.0}$_{\pm0.0}$ \\ GunPointAgeSpan & 0.9262$_{\pm0.0037}$ & 0.8755$_{\pm0.0174}$ & 0.9916$_{\pm0.0018}$ & 0.9662$_{\pm0.0048}$ & \textbf{0.9979}$_{\pm0.0037}$ \\ GunPointMaleVersusFemale & 0.9905$_{\pm0.0032}$ & 0.961$_{\pm0.0391}$ & 0.9989$_{\pm0.0018}$ & 0.9916$_{\pm0.0048}$ & \textbf{1.0}$_{\pm0.0}$ \\ GunPointOldVersusYoung & 0.9545$_{\pm0.0183}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.982$_{\pm0.0048}$ & \textbf{1.0}$_{\pm0.0}$ \\ Ham & 0.7556$_{\pm0.011}$ & 0.6698$_{\pm0.024}$ & 0.7238$_{\pm0.019}$ & \textbf{0.7746}$_{\pm0.0306}$ & 0.7238$_{\pm0.0415}$ \\ Haptics & 0.4069$_{\pm0.0315}$ & 0.4426$_{\pm0.0167}$ & 0.487$_{\pm0.0142}$ & 0.4167$_{\pm0.0179}$ & \textbf{0.5227}$_{\pm0.0149}$ \\ Herring & 0.5677$_{\pm0.113}$ & \textbf{0.6458}$_{\pm0.0325}$ & \textbf{0.6458}$_{\pm0.0393}$ & 0.5677$_{\pm0.0325}$ & 0.6354$_{\pm0.0325}$ \\ HouseTwenty & 0.8347$_{\pm0.0097}$ & 0.8123$_{\pm0.0049}$ & 0.9272$_{\pm0.0097}$ & 0.8992$_{\pm0.0084}$ & \textbf{0.9804}$_{\pm0.0049}$ \\ InsectEPGRegularTrain & 0.7068$_{\pm0.0106}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.9331$_{\pm0.0267}$ & \textbf{1.0}$_{\pm0.0}$ \\ \midrule Blink & 0.9474$_{\pm0.02}$ & 0.9193$_{\pm0.0161}$ & 0.9163$_{\pm0.0334}$ & 0.9963$_{\pm0.0013}$ & \textbf{0.9978}$_{\pm0.0}$ \\ SharePriceIncrease & 0.6425$_{\pm0.0016}$ & 0.6763$_{\pm0.015}$ & 0.6446$_{\pm0.0237}$ & \textbf{0.6863}$_{\pm0.0}$ & 0.6408$_{\pm0.0187}$ \\ \midrule ArticularyWordRecognition & 0.9344$_{\pm0.0069}$ & 0.9633$_{\pm0.0058}$ & 0.9833$_{\pm0.0033}$ & 0.9733$_{\pm0.0033}$ & \textbf{0.9944}$_{\pm0.0019}$ \\ BasicMotions & 0.85$_{\pm0.025}$ & 0.8833$_{\pm0.0144}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ CharacterTrajectories & 0.9666$_{\pm0.0014}$ & 0.9731$_{\pm0.0056}$ & 0.98$_{\pm0.0036}$ & 0.9791$_{\pm0.0028}$ & \textbf{0.9893}$_{\pm0.0008}$ \\ Cricket & 0.8472$_{\pm0.0605}$ & 0.9213$_{\pm0.008}$ & 0.9907$_{\pm0.008}$ & 0.9491$_{\pm0.016}$ & \textbf{0.9954}$_{\pm0.008}$ \\ ERing & 0.779$_{\pm0.0483}$ & 0.9074$_{\pm0.0098}$ & 0.9506$_{\pm0.0077}$ & 0.9358$_{\pm0.0077}$ & \textbf{0.9827}$_{\pm0.0043}$ \\ Epilepsy & 0.9348$_{\pm0.0192}$ & 0.8092$_{\pm0.0233}$ & 0.9952$_{\pm0.0042}$ & 0.9976$_{\pm0.0042}$ & \textbf{1.0}$_{\pm0.0}$ \\ HandMovementDirection & 0.3378$_{\pm0.027}$ & \textbf{0.4685}$_{\pm0.068}$ & 0.3153$_{\pm0.0281}$ & 0.3108$_{\pm0.0358}$ & 0.3108$_{\pm0.0702}$ \\ \bottomrule \end{tabular} }`{=latex} ```{=latex} \hfill ``` `\scalebox{\scalefactorsotaft}{ \begin{tabular}{l|lllll} \toprule & UniTS & GPT4TS & NuTime & MOMENT & Mantis \\ \midrule InsectEPGSmallTrain & 0.6667$_{\pm0.0145}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.8969$_{\pm0.0023}$ & \textbf{1.0}$_{\pm0.0}$ \\ InsectWingbeatSound & 0.5726$_{\pm0.016}$ & 0.6165$_{\pm0.0069}$ & \textbf{0.619}$_{\pm0.0048}$ & 0.6019$_{\pm0.0034}$ & 0.5961$_{\pm0.0051}$ \\ ItalyPowerDemand & 0.964$_{\pm0.0019}$ & 0.9637$_{\pm0.0011}$ & \textbf{0.9666}$_{\pm0.002}$ & 0.9624$_{\pm0.0006}$ & 0.9624$_{\pm0.0015}$ \\ LargeKitchenAppliances & 0.6942$_{\pm0.0178}$ & 0.5156$_{\pm0.0285}$ & 0.824$_{\pm0.0116}$ & \textbf{0.8693}$_{\pm0.0232}$ & 0.8516$_{\pm0.0111}$ \\ Lightning2 & 0.7486$_{\pm0.0663}$ & 0.7268$_{\pm0.025}$ & 0.776$_{\pm0.025}$ & 0.7978$_{\pm0.025}$ & \textbf{0.8087}$_{\pm0.0189}$ \\ Lightning7 & 0.5571$_{\pm0.0209}$ & 0.5982$_{\pm0.0079}$ & 0.7032$_{\pm0.0519}$ & \textbf{0.7717}$_{\pm0.0209}$ & 0.7397$_{\pm0.0362}$ \\ Mallat & 0.8736$_{\pm0.011}$ & 0.92$_{\pm0.0258}$ & \textbf{0.9508}$_{\pm0.0191}$ & 0.9205$_{\pm0.026}$ & 0.9404$_{\pm0.0072}$ \\ Meat & 0.9111$_{\pm0.0255}$ & 0.6944$_{\pm0.2269}$ & \textbf{0.9611}$_{\pm0.0255}$ & 0.6444$_{\pm0.0674}$ & 0.9333$_{\pm0.0289}$ \\ MedicalImages & 0.6553$_{\pm0.0026}$ & 0.611$_{\pm0.0201}$ & 0.7513$_{\pm0.0149}$ & 0.7206$_{\pm0.029}$ & \textbf{0.7662}$_{\pm0.0178}$ \\ MelbournePedestrian & 0.8711$_{\pm0.0025}$ & 0.9351$_{\pm0.0037}$ & \textbf{0.9601}$_{\pm0.0019}$ & 0.8795$_{\pm0.0055}$ & 0.9552$_{\pm0.001}$ \\ MiddlePhalanxOutlineAgeGroup & 0.6039$_{\pm0.0065}$ & 0.6104$_{\pm0.0065}$ & 0.5368$_{\pm0.0198}$ & \textbf{0.6126}$_{\pm0.0327}$ & 0.5996$_{\pm0.0209}$ \\ MiddlePhalanxOutlineCorrect & 0.827$_{\pm0.0241}$ & 0.6518$_{\pm0.0436}$ & 0.7938$_{\pm0.0248}$ & 0.8076$_{\pm0.0034}$ & \textbf{0.8339}$_{\pm0.0139}$ \\ MiddlePhalanxTW & 0.5779$_{\pm0.0234}$ & \textbf{0.5909}$_{\pm0.0}$ & 0.4632$_{\pm0.015}$ & 0.5606$_{\pm0.0037}$ & 0.4827$_{\pm0.03}$ \\ MixedShapesSmallTrain & 0.7281$_{\pm0.0569}$ & 0.8115$_{\pm0.0111}$ & 0.9281$_{\pm0.0054}$ & 0.8357$_{\pm0.0363}$ & \textbf{0.9531}$_{\pm0.0038}$ \\ MoteStrain & 0.8339$_{\pm0.0081}$ & 0.8243$_{\pm0.0062}$ & \textbf{0.9609}$_{\pm0.0037}$ & 0.8341$_{\pm0.0264}$ & 0.9068$_{\pm0.0174}$ \\ NonInvasiveFetalECGThorax1 & 0.8879$_{\pm0.0059}$ & 0.7774$_{\pm0.0152}$ & \textbf{0.9342}$_{\pm0.0076}$ & 0.7535$_{\pm0.0237}$ & 0.9086$_{\pm0.0063}$ \\ NonInvasiveFetalECGThorax2 & 0.9137$_{\pm0.0015}$ & 0.839$_{\pm0.0098}$ & \textbf{0.9381}$_{\pm0.0046}$ & 0.7995$_{\pm0.0033}$ & 0.9216$_{\pm0.0055}$ \\ OSULeaf & 0.5482$_{\pm0.0391}$ & 0.4669$_{\pm0.018}$ & 0.8967$_{\pm0.0286}$ & 0.8017$_{\pm0.0219}$ & \textbf{0.9642}$_{\pm0.0024}$ \\ OliveOil & 0.4778$_{\pm0.0385}$ & 0.4889$_{\pm0.0385}$ & 0.7778$_{\pm0.0385}$ & 0.4667$_{\pm0.0667}$ & \textbf{0.8889}$_{\pm0.0509}$ \\ PhalangesOutlinesCorrect & 0.8015$_{\pm0.0037}$ & 0.6437$_{\pm0.0041}$ & \textbf{0.8263}$_{\pm0.0031}$ & 0.803$_{\pm0.0047}$ & 0.8162$_{\pm0.0128}$ \\ Phoneme & 0.1431$_{\pm0.0086}$ & 0.1466$_{\pm0.0023}$ & 0.2973$_{\pm0.0081}$ & 0.2904$_{\pm0.009}$ & \textbf{0.3214}$_{\pm0.0059}$ \\ PickupGestureWiimoteZ & 0.56$_{\pm0.02}$ & 0.7133$_{\pm0.0231}$ & 0.7267$_{\pm0.0115}$ & 0.72$_{\pm0.0}$ & \textbf{0.76}$_{\pm0.0529}$ \\ Plane & 0.9524$_{\pm0.0165}$ & 0.9778$_{\pm0.0055}$ & \textbf{1.0}$_{\pm0.0}$ & 0.9873$_{\pm0.0145}$ & \textbf{1.0}$_{\pm0.0}$ \\ PowerCons & 0.95$_{\pm0.02}$ & \textbf{1.0}$_{\pm0.0}$ & 0.9889$_{\pm0.0111}$ & 0.9167$_{\pm0.0111}$ & 0.9944$_{\pm0.0096}$ \\ ProximalPhalanxOutlineAgeGroup & 0.8407$_{\pm0.0157}$ & 0.8016$_{\pm0.0203}$ & \textbf{0.8553}$_{\pm0.0102}$ & 0.8455$_{\pm0.0123}$ & 0.8455$_{\pm0.0246}$ \\ ProximalPhalanxOutlineCorrect & 0.8568$_{\pm0.0052}$ & 0.8179$_{\pm0.0034}$ & \textbf{0.9072}$_{\pm0.006}$ & 0.8259$_{\pm0.0221}$ & 0.8763$_{\pm0.0209}$ \\ ProximalPhalanxTW & 0.7951$_{\pm0.0098}$ & 0.7967$_{\pm0.0102}$ & \textbf{0.8179}$_{\pm0.0056}$ & 0.787$_{\pm0.0102}$ & 0.7772$_{\pm0.0314}$ \\ RefrigerationDevices & 0.4658$_{\pm0.0248}$ & 0.4453$_{\pm0.0092}$ & \textbf{0.5511}$_{\pm0.0227}$ & 0.4996$_{\pm0.0336}$ & 0.4916$_{\pm0.0111}$ \\ Rock & 0.6133$_{\pm0.0945}$ & 0.5933$_{\pm0.0503}$ & 0.68$_{\pm0.02}$ & 0.6733$_{\pm0.0611}$ & \textbf{0.7133}$_{\pm0.0231}$ \\ ScreenType & 0.3556$_{\pm0.0041}$ & 0.3956$_{\pm0.0126}$ & \textbf{0.5289}$_{\pm0.0434}$ & 0.4213$_{\pm0.0116}$ & 0.4782$_{\pm0.0041}$ \\ ShakeGestureWiimoteZ & 0.6067$_{\pm0.0416}$ & 0.6867$_{\pm0.0115}$ & 0.9133$_{\pm0.0231}$ & 0.9067$_{\pm0.0115}$ & \textbf{0.92}$_{\pm0.0}$ \\ ShapeletSim & 0.6981$_{\pm0.0651}$ & 0.5074$_{\pm0.0128}$ & 0.8852$_{\pm0.0534}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ ShapesAll & 0.67$_{\pm0.012}$ & 0.6833$_{\pm0.0029}$ & 0.8856$_{\pm0.0067}$ & 0.8039$_{\pm0.0108}$ & \textbf{0.8906}$_{\pm0.0059}$ \\ SmallKitchenAppliances & 0.6276$_{\pm0.032}$ & 0.5529$_{\pm0.0171}$ & 0.7822$_{\pm0.0015}$ & 0.6756$_{\pm0.0161}$ & \textbf{0.7929}$_{\pm0.0031}$ \\ SmoothSubspace & 0.8156$_{\pm0.0102}$ & 0.8911$_{\pm0.0038}$ & 0.9889$_{\pm0.0038}$ & 0.7733$_{\pm0.0291}$ & \textbf{0.9978}$_{\pm0.0038}$ \\ SonyAIBORobotSurface1 & 0.721$_{\pm0.0846}$ & 0.6267$_{\pm0.0215}$ & \textbf{0.8525}$_{\pm0.0189}$ & 0.7604$_{\pm0.0204}$ & 0.8142$_{\pm0.0113}$ \\ SonyAIBORobotSurface2 & 0.829$_{\pm0.0252}$ & 0.8454$_{\pm0.0205}$ & 0.8506$_{\pm0.0115}$ & 0.8765$_{\pm0.0276}$ & \textbf{0.8996}$_{\pm0.0112}$ \\ Strawberry & 0.9577$_{\pm0.0172}$ & 0.7261$_{\pm0.0788}$ & \textbf{0.9712}$_{\pm0.0031}$ & 0.9459$_{\pm0.0054}$ & 0.9667$_{\pm0.0078}$ \\ SwedishLeaf & 0.88$_{\pm0.0085}$ & 0.8203$_{\pm0.0305}$ & 0.9573$_{\pm0.0024}$ & 0.9376$_{\pm0.007}$ & \textbf{0.9712}$_{\pm0.0042}$ \\ Symbols & 0.8211$_{\pm0.0348}$ & 0.8003$_{\pm0.0243}$ & 0.9779$_{\pm0.0053}$ & 0.9481$_{\pm0.0192}$ & \textbf{0.9889}$_{\pm0.0017}$ \\ SyntheticControl & 0.95$_{\pm0.0088}$ & 0.9622$_{\pm0.0117}$ & 0.9889$_{\pm0.0051}$ & 0.9889$_{\pm0.0019}$ & \textbf{0.9944}$_{\pm0.0038}$ \\ ToeSegmentation1 & 0.8012$_{\pm0.0373}$ & 0.5687$_{\pm0.0051}$ & 0.9459$_{\pm0.0101}$ & 0.9284$_{\pm0.0483}$ & \textbf{0.9664}$_{\pm0.0067}$ \\ ToeSegmentation2 & 0.7974$_{\pm0.016}$ & 0.6923$_{\pm0.0353}$ & 0.8974$_{\pm0.0089}$ & \textbf{0.9308}$_{\pm0.0204}$ & 0.9231$_{\pm0.0077}$ \\ Trace & 0.9467$_{\pm0.0208}$ & 0.8567$_{\pm0.0577}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ TwoLeadECG & 0.7548$_{\pm0.0388}$ & 0.7934$_{\pm0.0272}$ & 0.9104$_{\pm0.0273}$ & 0.8879$_{\pm0.0509}$ & \textbf{0.9941}$_{\pm0.0018}$ \\ TwoPatterns & 0.9811$_{\pm0.0043}$ & 0.9676$_{\pm0.0173}$ & 0.9998$_{\pm0.0001}$ & 0.9998$_{\pm0.0002}$ & \textbf{0.9999}$_{\pm0.0001}$ \\ UMD & 0.8056$_{\pm0.0751}$ & 0.9699$_{\pm0.008}$ & \textbf{1.0}$_{\pm0.0}$ & 0.9815$_{\pm0.02}$ & 0.9907$_{\pm0.004}$ \\ UWaveGestureLibraryX & 0.7635$_{\pm0.0037}$ & 0.7319$_{\pm0.0069}$ & 0.8516$_{\pm0.0044}$ & 0.8062$_{\pm0.0076}$ & \textbf{0.8679}$_{\pm0.0028}$ \\ UWaveGestureLibraryY & 0.6663$_{\pm0.0022}$ & 0.6145$_{\pm0.0027}$ & 0.7701$_{\pm0.005}$ & 0.7378$_{\pm0.0127}$ & \textbf{0.7917}$_{\pm0.0073}$ \\ UWaveGestureLibraryZ & 0.6903$_{\pm0.0045}$ & 0.6224$_{\pm0.0217}$ & 0.7813$_{\pm0.0073}$ & 0.757$_{\pm0.0158}$ & \textbf{0.8053}$_{\pm0.0011}$ \\ Wafer & 0.9951$_{\pm0.0009}$ & 0.9948$_{\pm0.0022}$ & 0.9987$_{\pm0.0003}$ & 0.9982$_{\pm0.0004}$ & \textbf{0.9991}$_{\pm0.0003}$ \\ Wine & 0.5185$_{\pm0.0321}$ & 0.5185$_{\pm0.0321}$ & \textbf{0.8025}$_{\pm0.0466}$ & 0.5$_{\pm0.0}$ & 0.6605$_{\pm0.0107}$ \\ WordSynonyms & 0.5225$_{\pm0.0086}$ & 0.5397$_{\pm0.0096}$ & 0.6917$_{\pm0.0065}$ & 0.7079$_{\pm0.022}$ & \textbf{0.7429}$_{\pm0.0031}$ \\ Worms & 0.5455$_{\pm0.0344}$ & 0.4416$_{\pm0.0225}$ & \textbf{0.7229}$_{\pm0.0198}$ & 0.7013$_{\pm0.0225}$ & 0.7143$_{\pm0.026}$ \\ WormsTwoClass & 0.5931$_{\pm0.0417}$ & 0.6364$_{\pm0.013}$ & 0.7143$_{\pm0.0225}$ & 0.7532$_{\pm0.045}$ & \textbf{0.8052}$_{\pm0.0344}$ \\ Yoga & 0.7931$_{\pm0.0094}$ & 0.7467$_{\pm0.0025}$ & 0.8053$_{\pm0.0118}$ & 0.8477$_{\pm0.0057}$ & \textbf{0.8709}$_{\pm0.0132}$ \\ \midrule Handwriting & 0.1537$_{\pm0.0198}$ & 0.2922$_{\pm0.0049}$ & 0.2545$_{\pm0.0118}$ & 0.3635$_{\pm0.0101}$ & \textbf{0.4631}$_{\pm0.0147}$ \\ JapaneseVowels & 0.9279$_{\pm0.0102}$ & \textbf{0.9775}$_{\pm0.0041}$ & 0.9486$_{\pm0.0047}$ & 0.9486$_{\pm0.0027}$ & 0.9694$_{\pm0.0056}$ \\ LSST & 0.4862$_{\pm0.02}$ & 0.108$_{\pm0.0611}$ & 0.3335$_{\pm0.0282}$ & 0.5827$_{\pm0.0162}$ & \textbf{0.6035}$_{\pm0.0254}$ \\ Libras & 0.6852$_{\pm0.014}$ & 0.7333$_{\pm0.0096}$ & 0.8407$_{\pm0.0064}$ & 0.7852$_{\pm0.0064}$ & \textbf{0.8722}$_{\pm0.0056}$ \\ NATOPS & 0.7926$_{\pm0.0225}$ & \textbf{0.9537}$_{\pm0.0064}$ & 0.863$_{\pm0.021}$ & 0.8593$_{\pm0.0195}$ & 0.9315$_{\pm0.0085}$ \\ PhonemeSpectra & 0.1121$_{\pm0.0067}$ & 0.1106$_{\pm0.0054}$ & 0.2392$_{\pm0.007}$ & 0.1815$_{\pm0.0111}$ & \textbf{0.3218}$_{\pm0.0027}$ \\ RacketSports & 0.7719$_{\pm0.0325}$ & 0.7961$_{\pm0.0174}$ & 0.8794$_{\pm0.0137}$ & 0.7895$_{\pm0.0132}$ & \textbf{0.9189}$_{\pm0.01}$ \\ SpokenArabicDigits & 0.9782$_{\pm0.0021}$ & 0.9856$_{\pm0.0023}$ & 0.9853$_{\pm0.0009}$ & 0.9818$_{\pm0.0014}$ & \textbf{0.9867}$_{\pm0.0019}$ \\ UWaveGestureLibrary & 0.8167$_{\pm0.0048}$ & 0.8396$_{\pm0.0065}$ & 0.8646$_{\pm0.0188}$ & 0.9156$_{\pm0.0094}$ & \textbf{0.9281}$_{\pm0.0125}$ \\ \midrule \textit{\textbf{Best Count}} & 2 & 13 & 38 & 14 & \textbf{81} \\ \bottomrule \end{tabular} }`{=latex} ```{=latex} \centering ``` `\scalebox{\scalefactorzeroshot}{ \begin{tabular}{l|llll} \toprule & Mantis w/o Diff & Mantis w/ Diff \\ \midrule ACSF1 & \textbf{0.6433}$_{\pm0.0252}$ & 0.6133$_{\pm0.0208}$ \\ Adiac & 0.6931$_{\pm0.0068}$ & \textbf{0.7332}$_{\pm0.0039}$ \\ AllGestureWiimoteX & 0.641$_{\pm0.0054}$ & \textbf{0.6705}$_{\pm0.0044}$ \\ AllGestureWiimoteY & 0.6343$_{\pm0.0071}$ & \textbf{0.6671}$_{\pm0.0057}$ \\ AllGestureWiimoteZ & 0.631$_{\pm0.0068}$ & \textbf{0.6695}$_{\pm0.0033}$ \\ ArrowHead & \textbf{0.7543}$_{\pm0.0057}$ & 0.7105$_{\pm0.0175}$ \\ BME & 0.8689$_{\pm0.0038}$ & \textbf{0.9311}$_{\pm0.0038}$ \\ Beef & 0.6$_{\pm0.0}$ & \textbf{0.6556}$_{\pm0.0509}$ \\ BeetleFly & \textbf{0.9}$_{\pm0.05}$ & 0.85$_{\pm0.05}$ \\ BirdChicken & 0.8833$_{\pm0.0289}$ & \textbf{1.0}$_{\pm0.0}$ \\ CBF & \textbf{0.9937}$_{\pm0.0017}$ & 0.993$_{\pm0.0013}$ \\ Car & 0.7389$_{\pm0.0096}$ & \textbf{0.7722}$_{\pm0.0419}$ \\ Chinatown & 0.8678$_{\pm0.0131}$ & \textbf{0.8737}$_{\pm0.0168}$ \\ ChlorineConcentration & 0.6729$_{\pm0.0017}$ & \textbf{0.6806}$_{\pm0.0019}$ \\ CinCECGTorso & \textbf{0.6664}$_{\pm0.0098}$ & 0.6611$_{\pm0.0036}$ \\ Coffee & 0.9286$_{\pm0.0}$ & \textbf{0.9524}$_{\pm0.0206}$ \\ Computers & 0.7267$_{\pm0.0046}$ & \textbf{0.7373}$_{\pm0.0092}$ \\ CricketX & 0.706$_{\pm0.0146}$ & \textbf{0.7368}$_{\pm0.0171}$ \\ CricketY & 0.7496$_{\pm0.0107}$ & \textbf{0.7504}$_{\pm0.0065}$ \\ CricketZ & 0.7889$_{\pm0.003}$ & \textbf{0.7906}$_{\pm0.0039}$ \\ Crop & \textbf{0.6879}$_{\pm0.0009}$ & 0.6756$_{\pm0.0018}$ \\ DiatomSizeReduction & 0.7963$_{\pm0.0136}$ & \textbf{0.8845}$_{\pm0.0019}$ \\ DistalPhalanxOutlineAgeGroup & 0.7482$_{\pm0.0125}$ & \textbf{0.789}$_{\pm0.015}$ \\ DistalPhalanxOutlineCorrect & \textbf{0.7681}$_{\pm0.0036}$ & 0.75$_{\pm0.0126}$ \\ DistalPhalanxTW & 0.6571$_{\pm0.0042}$ & \textbf{0.6859}$_{\pm0.011}$ \\ DodgerLoopDay & \textbf{0.5583}$_{\pm0.0191}$ & 0.55$_{\pm0.0217}$ \\ DodgerLoopGame & 0.6618$_{\pm0.0274}$ & \textbf{0.7585}$_{\pm0.0221}$ \\ DodgerLoopWeekend & \textbf{0.9758}$_{\pm0.0042}$ & 0.9517$_{\pm0.0084}$ \\ ECG200 & \textbf{0.86}$_{\pm0.02}$ & 0.82$_{\pm0.01}$ \\ ECG5000 & \textbf{0.9283}$_{\pm0.0011}$ & 0.9211$_{\pm0.001}$ \\ ECGFiveDays & 0.8668$_{\pm0.0159}$ & \textbf{0.909}$_{\pm0.0218}$ \\ EOGHorizontalSignal & \textbf{0.5875}$_{\pm0.0188}$ & \textbf{0.5875}$_{\pm0.0089}$ \\ EOGVerticalSignal & 0.4365$_{\pm0.0127}$ & \textbf{0.4751}$_{\pm0.0}$ \\ Earthquakes & \textbf{0.753}$_{\pm0.0042}$ & 0.7482$_{\pm0.0}$ \\ ElectricDevices & 0.7045$_{\pm0.0006}$ & \textbf{0.7226}$_{\pm0.0026}$ \\ EthanolLevel & 0.2873$_{\pm0.0117}$ & \textbf{0.2993}$_{\pm0.011}$ \\ FaceAll & \textbf{0.7992}$_{\pm0.003}$ & 0.7815$_{\pm0.0074}$ \\ FaceFour & 0.947$_{\pm0.0131}$ & \textbf{0.9508}$_{\pm0.0066}$ \\ FacesUCR & \textbf{0.8364}$_{\pm0.0067}$ & 0.8354$_{\pm0.0054}$ \\ FiftyWords & 0.5758$_{\pm0.0066}$ & \textbf{0.6462}$_{\pm0.0096}$ \\ Fish & 0.9124$_{\pm0.0132}$ & \textbf{0.9333}$_{\pm0.0066}$ \\ FordA & 0.8328$_{\pm0.0012}$ & \textbf{0.8581}$_{\pm0.0048}$ \\ FordB & 0.7198$_{\pm0.0096}$ & \textbf{0.7305}$_{\pm0.0031}$ \\ FreezerRegularTrain & \textbf{0.9533}$_{\pm0.0013}$ & 0.9374$_{\pm0.0043}$ \\ FreezerSmallTrain & 0.7556$_{\pm0.004}$ & \textbf{0.7942}$_{\pm0.0059}$ \\ Fungi & 0.81$_{\pm0.0062}$ & \textbf{0.8262}$_{\pm0.0164}$ \\ GestureMidAirD1 & 0.6359$_{\pm0.0194}$ & \textbf{0.659}$_{\pm0.0044}$ \\ GestureMidAirD2 & \textbf{0.6308}$_{\pm0.0077}$ & 0.6154$_{\pm0.0077}$ \\ GestureMidAirD3 & \textbf{0.341}$_{\pm0.016}$ & 0.3282$_{\pm0.016}$ \\ GesturePebbleZ1 & \textbf{0.9302}$_{\pm0.0}$ & 0.9283$_{\pm0.0034}$ \\ GesturePebbleZ2 & 0.9051$_{\pm0.011}$ & \textbf{0.9219}$_{\pm0.0256}$ \\ GunPoint & 0.9711$_{\pm0.0038}$ & \textbf{0.98}$_{\pm0.0067}$ \\ GunPointAgeSpan & 0.9736$_{\pm0.0018}$ & \textbf{0.9905}$_{\pm0.0}$ \\ GunPointMaleVersusFemale & 0.9937$_{\pm0.0}$ & \textbf{0.9958}$_{\pm0.0018}$ \\ GunPointOldVersusYoung & \textbf{0.9968}$_{\pm0.0}$ & \textbf{0.9968}$_{\pm0.0}$ \\ Ham & 0.6$_{\pm0.0095}$ & \textbf{0.673}$_{\pm0.0145}$ \\ HandOutlines & 0.9063$_{\pm0.0056}$ & \textbf{0.9162}$_{\pm0.0072}$ \\ Haptics & 0.4448$_{\pm0.0162}$ & \textbf{0.4968}$_{\pm0.0032}$ \\ Herring & \textbf{0.6979}$_{\pm0.018}$ & 0.6667$_{\pm0.0239}$ \\ HouseTwenty & \textbf{0.944}$_{\pm0.0049}$ & 0.9412$_{\pm0.0}$ \\ InlineSkate & \textbf{0.3703}$_{\pm0.0107}$ & 0.363$_{\pm0.0136}$ \\ InsectEPGRegularTrain & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ InsectEPGSmallTrain & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ InsectWingbeatSound & 0.4739$_{\pm0.006}$ & \textbf{0.519}$_{\pm0.0044}$ \\ \midrule Blink & 0.9948$_{\pm0.0013}$ & \textbf{0.9993}$_{\pm0.0013}$ \\ EMOPain & \textbf{0.8423}$_{\pm0.0056}$ & 0.8385$_{\pm0.0016}$ \\ MotionSenseHAR & 0.9925$_{\pm0.0038}$ & \textbf{0.9975}$_{\pm0.0022}$\\ SharePriceIncrease & \textbf{0.6877}$_{\pm0.0026}$ & 0.6836$_{\pm0.0049}$ \\ \midrule ArticularyWordRecognition & 0.9911$_{\pm0.0019}$ & \textbf{0.993}$_{\pm0.003}$ \\ BasicMotions & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ CharacterTrajectories & 0.9352$_{\pm0.0025}$ & \textbf{0.94}$_{\pm0.001}$ \\ Cricket & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ DuckDuckGeese & 0.38$_{\pm0.0529}$ & \textbf{0.407}$_{\pm0.031}$ \\ ERing & 0.8284$_{\pm0.0311}$ & \textbf{0.941}$_{\pm0.01}$ \\ EigenWorms & \textbf{0.7557}$_{\pm0.0076}$ & 0.753$_{\pm0.016}$ \\ Epilepsy & 0.9928$_{\pm0.0072}$ & \textbf{0.995}$_{\pm0.004}$ \\ EthanolConcentration & \textbf{0.3308}$_{\pm0.0076}$ & 0.27$_{\pm0.01}$ \\ FaceDetection & 0.5138$_{\pm0.0078}$ & \textbf{0.526}$_{\pm0.002}$ \\ FingerMovements & 0.5333$_{\pm0.0058}$ & \textbf{0.54}$_{\pm0.035}$ \\ HandMovementDirection & \textbf{0.3108}$_{\pm0.0135}$ & 0.212$_{\pm0.041}$ \\ \bottomrule \end{tabular} }`{=latex} ```{=latex} \hfill ``` ```{=latex} \centering ``` `\scalebox{\scalefactorzeroshot}{ \begin{tabular}{l|llll} \toprule & Mantis w/o Diff & Mantis w/ Diff \\ \midrule ItalyPowerDemand & 0.8996$_{\pm0.003}$ & \textbf{0.9077}$_{\pm0.0035}$ \\ LargeKitchenAppliances & 0.7662$_{\pm0.0094}$ & \textbf{0.7804}$_{\pm0.0041}$ \\ Lightning2 & 0.7268$_{\pm0.0189}$ & \textbf{0.8033}$_{\pm0.0}$ \\ Lightning7 & 0.6941$_{\pm0.0209}$ & \textbf{0.7763}$_{\pm0.0285}$ \\ Mallat & \textbf{0.8904}$_{\pm0.0074}$ & 0.8903$_{\pm0.0137}$ \\ Meat & \textbf{0.95}$_{\pm0.0167}$ & 0.9389$_{\pm0.0096}$ \\ MedicalImages & 0.682$_{\pm0.0033}$ & \textbf{0.7079}$_{\pm0.0035}$ \\ MelbournePedestrian & \textbf{0.9083}$_{\pm0.0018}$ & 0.9016$_{\pm0.0031}$ \\ MiddlePhalanxOutlineAgeGroup & \textbf{0.5952}$_{\pm0.0163}$ & 0.5801$_{\pm0.0099}$ \\ MiddlePhalanxOutlineCorrect & 0.8087$_{\pm0.004}$ & \textbf{0.8099}$_{\pm0.0099}$ \\ MiddlePhalanxTW & \textbf{0.5433}$_{\pm0.0099}$ & 0.5368$_{\pm0.0099}$ \\ MixedShapesRegularTrain & 0.8995$_{\pm0.0017}$ & \textbf{0.943}$_{\pm0.0044}$ \\ MixedShapesSmallTrain & 0.8476$_{\pm0.0069}$ & \textbf{0.8961}$_{\pm0.0004}$ \\ MoteStrain & \textbf{0.9255}$_{\pm0.0058}$ & 0.9137$_{\pm0.0136}$ \\ NonInvasiveFetalECGThorax1 & \textbf{0.7837}$_{\pm0.0005}$ & 0.6222$_{\pm0.0037}$ \\ NonInvasiveFetalECGThorax2 & \textbf{0.8053}$_{\pm0.0043}$ & 0.6872$_{\pm0.0025}$ \\ OSULeaf & 0.7617$_{\pm0.0048}$ & \textbf{0.8747}$_{\pm0.0104}$ \\ OliveOil & 0.8889$_{\pm0.0192}$ & \textbf{0.9333}$_{\pm0.0}$ \\ PLAID & \textbf{0.8442}$_{\pm0.0043}$ & 0.8181$_{\pm0.0047}$ \\ PhalangesOutlinesCorrect & \textbf{0.7949}$_{\pm0.0031}$ & 0.7786$_{\pm0.0042}$ \\ Phoneme & 0.289$_{\pm0.0024}$ & \textbf{0.323}$_{\pm0.0034}$ \\ PickupGestureWiimoteZ & 0.7$_{\pm0.02}$ & \textbf{0.74}$_{\pm0.02}$ \\ PigAirwayPressure & 0.4663$_{\pm0.0}$ & \textbf{0.484}$_{\pm0.0147}$ \\ PigArtPressure & 0.9071$_{\pm0.0028}$ & \textbf{0.9103}$_{\pm0.0028}$ \\ PigCVP & \textbf{0.8269}$_{\pm0.0048}$ & 0.7837$_{\pm0.0127}$ \\ Plane & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ PowerCons & \textbf{0.9315}$_{\pm0.014}$ & 0.9093$_{\pm0.0032}$ \\ ProximalPhalanxOutlineAgeGroup & \textbf{0.8537}$_{\pm0.0049}$ & \textbf{0.8537}$_{\pm0.0049}$ \\ ProximalPhalanxOutlineCorrect & \textbf{0.8213}$_{\pm0.0034}$ & 0.8202$_{\pm0.0086}$ \\ ProximalPhalanxTW & \textbf{0.7707}$_{\pm0.0098}$ & 0.7691$_{\pm0.0028}$ \\ RefrigerationDevices & \textbf{0.5138}$_{\pm0.0101}$ & 0.504$_{\pm0.0122}$ \\ Rock & 0.6933$_{\pm0.0808}$ & \textbf{0.7133}$_{\pm0.0115}$ \\ ScreenType & \textbf{0.5031}$_{\pm0.0101}$ & 0.4649$_{\pm0.0134}$ \\ SemgHandGenderCh2 & 0.8772$_{\pm0.0051}$ & \textbf{0.9189}$_{\pm0.0086}$ \\ SemgHandMovementCh2 & 0.7622$_{\pm0.0059}$ & \textbf{0.797}$_{\pm0.0056}$ \\ SemgHandSubjectCh2 & 0.8304$_{\pm0.0114}$ & \textbf{0.8622}$_{\pm0.0135}$ \\ ShakeGestureWiimoteZ & 0.8733$_{\pm0.0115}$ & \textbf{0.8867}$_{\pm0.0115}$ \\ ShapeletSim & \textbf{0.963}$_{\pm0.0032}$ & 0.9278$_{\pm0.0111}$ \\ ShapesAll & 0.8189$_{\pm0.0107}$ & \textbf{0.8194}$_{\pm0.0054}$ \\ SmallKitchenAppliances & 0.8036$_{\pm0.0094}$ & \textbf{0.8089}$_{\pm0.0081}$ \\ SmoothSubspace & \textbf{0.9289}$_{\pm0.0102}$ & 0.9067$_{\pm0.0115}$ \\ SonyAIBORobotSurface1 & 0.7682$_{\pm0.02}$ & \textbf{0.787}$_{\pm0.0294}$ \\ SonyAIBORobotSurface2 & 0.7964$_{\pm0.0184}$ & \textbf{0.8552}$_{\pm0.0168}$ \\ StarLightCurves & \textbf{0.9772}$_{\pm0.0005}$ & 0.9759$_{\pm0.0005}$ \\ Strawberry & \textbf{0.9541}$_{\pm0.0}$ & 0.9514$_{\pm0.0027}$ \\ SwedishLeaf & 0.9035$_{\pm0.0024}$ & \textbf{0.9275}$_{\pm0.0009}$ \\ Symbols & 0.9357$_{\pm0.0126}$ & \textbf{0.9698}$_{\pm0.0056}$ \\ SyntheticControl & 0.9478$_{\pm0.0051}$ & \textbf{0.9767}$_{\pm0.0067}$ \\ ToeSegmentation1 & 0.9415$_{\pm0.0154}$ & \textbf{0.9635}$_{\pm0.0067}$ \\ ToeSegmentation2 & 0.8718$_{\pm0.0364}$ & \textbf{0.9282}$_{\pm0.0044}$ \\ Trace & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ TwoLeadECG & 0.9839$_{\pm0.0056}$ & \textbf{0.9962}$_{\pm0.0005}$ \\ TwoPatterns & \textbf{0.8987}$_{\pm0.0034}$ & 0.8802$_{\pm0.0079}$ \\ UMD & 0.9491$_{\pm0.008}$ & \textbf{0.9722}$_{\pm0.0069}$ \\ UWaveGestureLibraryAll & 0.8283$_{\pm0.0036}$ & \textbf{0.8458}$_{\pm0.0057}$ \\ UWaveGestureLibraryX & 0.7487$_{\pm0.0012}$ & \textbf{0.7696}$_{\pm0.0028}$ \\ UWaveGestureLibraryY & 0.6676$_{\pm0.0017}$ & \textbf{0.6874}$_{\pm0.0022}$ \\ UWaveGestureLibraryZ & 0.7082$_{\pm0.0023}$ & \textbf{0.7324}$_{\pm0.0036}$ \\ Wafer & 0.9794$_{\pm0.0006}$ & \textbf{0.9903}$_{\pm0.0003}$ \\ Wine & 0.6728$_{\pm0.0107}$ & \textbf{0.7901}$_{\pm0.0107}$ \\ WordSynonyms & 0.4932$_{\pm0.0048}$ & \textbf{0.5502}$_{\pm0.0081}$ \\ Worms & 0.6017$_{\pm0.0075}$ & \textbf{0.6537}$_{\pm0.027}$ \\ WormsTwoClass & 0.6926$_{\pm0.0198}$ & \textbf{0.8139}$_{\pm0.0198}$ \\ Yoga & \textbf{0.8164}$_{\pm0.001}$ & 0.815$_{\pm0.0012}$ \\ \midrule Handwriting & 0.3027$_{\pm0.01}$ & \textbf{0.339}$_{\pm0.01}$ \\ Heartbeat & 0.7675$_{\pm0.0157}$ & \textbf{0.798}$_{\pm0.017}$ \\ InsectWingbeatSubset & 0.5837$_{\pm0.0064}$ & \textbf{0.607}$_{\pm0.007}$ \\ JapaneseVowels & 0.9514$_{\pm0.0047}$ & \textbf{0.963}$_{\pm0.006}$ \\ LSST & 0.5884$_{\pm0.0015}$ & \textbf{0.605}$_{\pm0.002}$ \\ Libras & 0.8815$_{\pm0.017}$ & \textbf{0.898}$_{\pm0.008}$ \\ MotorImagery & 0.5267$_{\pm0.0208}$ & \textbf{0.55}$_{\pm0.036}$ \\ NATOPS & 0.9$_{\pm0.0056}$ & \textbf{0.907}$_{\pm0.013}$ \\ PEMS-SF & 0.9807$_{\pm0.0067}$ & \textbf{0.985}$_{\pm0.013}$ \\ PhonemeSpectra & 0.2587$_{\pm0.0012}$ & \textbf{0.273}$_{\pm0.008}$ \\ RacketSports & 0.9123$_{\pm0.0137}$ & \textbf{0.923}$_{\pm0.004}$ \\ SelfRegulationSCP1 & 0.7611$_{\pm0.0034}$ & \textbf{0.804}$_{\pm0.011}$ \\ SelfRegulationSCP2 & \textbf{0.4796}$_{\pm0.0064}$ & 0.476$_{\pm0.045}$ \\ SpokenArabicDigits & 0.8172$_{\pm0.0081}$ & \textbf{0.839}$_{\pm0.005}$ \\ UWaveGestureLibrary & \textbf{0.8188}$_{\pm0.0083}$ & 0.814$_{\pm0.01}$ \\ \midrule \textit{\textbf{Best Count}} & 59 & \textbf{109} \\ \bottomrule \end{tabular} }`{=latex} ```{=latex} \newcommand{\scalefactorft}{0.55} ``` `\scalebox{\scalefactorft}{ \begin{tabular}{l|llll} \toprule & RF & Head & Scratch & Full \\ \midrule ACSF1 & 0.6133$_{\pm0.0208}$ & 0.6$_{\pm0.0}$ & 0.57$_{\pm0.0693}$ & \textbf{0.7033}$_{\pm0.0551}$ \\ Adiac & 0.7332$_{\pm0.0039}$ & 0.5857$_{\pm0.0051}$ & 0.7315$_{\pm0.0}$ & \textbf{0.8133}$_{\pm0.0}$ \\ AllGestureWiimoteX & 0.6705$_{\pm0.0044}$ & 0.5471$_{\pm0.0014}$ & 0.6838$_{\pm0.0073}$ & \textbf{0.7924}$_{\pm0.0073}$ \\ AllGestureWiimoteY & 0.6671$_{\pm0.0057}$ & 0.5595$_{\pm0.0044}$ & 0.7224$_{\pm0.0084}$ & \textbf{0.8033}$_{\pm0.0084}$ \\ AllGestureWiimoteZ & 0.6695$_{\pm0.0033}$ & 0.5414$_{\pm0.0014}$ & 0.6271$_{\pm0.0289}$ & \textbf{0.7443}$_{\pm0.0076}$ \\ ArrowHead & 0.7105$_{\pm0.0175}$ & 0.7276$_{\pm0.0119}$ & 0.7752$_{\pm0.0282}$ & \textbf{0.8114}$_{\pm0.0206}$ \\ BME & 0.9311$_{\pm0.0038}$ & 0.8756$_{\pm0.0139}$ & 0.9844$_{\pm0.0038}$ & \textbf{1.0}$_{\pm0.0}$ \\ Beef & 0.6556$_{\pm0.0509}$ & 0.6111$_{\pm0.0192}$ & 0.7667$_{\pm0.0882}$ & \textbf{0.7889}$_{\pm0.0192}$ \\ BeetleFly & 0.85$_{\pm0.05}$ & \textbf{0.9}$_{\pm0.0}$ & 0.5333$_{\pm0.0764}$ & 0.8167$_{\pm0.0289}$ \\ BirdChicken & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.8333$_{\pm0.0289}$ & \textbf{1.0}$_{\pm0.0}$ \\ CBF & 0.993$_{\pm0.0013}$ & 0.9941$_{\pm0.0006}$ & 0.9844$_{\pm0.0185}$ & \textbf{0.9993}$_{\pm0.0013}$ \\ Car & 0.7722$_{\pm0.0419}$ & 0.7778$_{\pm0.0192}$ & 0.7056$_{\pm0.0192}$ & \textbf{0.8833}$_{\pm0.0167}$ \\ Chinatown & 0.8737$_{\pm0.0168}$ & 0.8785$_{\pm0.0094}$ & 0.9582$_{\pm0.0067}$ & \textbf{0.9767}$_{\pm0.0077}$ \\ ChlorineConcentration & 0.6806$_{\pm0.0019}$ & 0.5688$_{\pm0.0011}$ & 0.7494$_{\pm0.0034}$ & \textbf{0.7718}$_{\pm0.01}$ \\ CinCECGTorso & 0.6611$_{\pm0.0036}$ & 0.7176$_{\pm0.0044}$ & 0.7802$_{\pm0.019}$ & \textbf{0.8539}$_{\pm0.0096}$ \\ Coffee & 0.9524$_{\pm0.0206}$ & 0.9643$_{\pm0.0}$ & 0.9881$_{\pm0.0206}$ & \textbf{1.0}$_{\pm0.0}$ \\ Computers & 0.7373$_{\pm0.0092}$ & 0.7093$_{\pm0.0129}$ & 0.7053$_{\pm0.022}$ & \textbf{0.7467}$_{\pm0.0205}$ \\ CricketX & 0.7368$_{\pm0.0171}$ & 0.6966$_{\pm0.003}$ & 0.7538$_{\pm0.0268}$ & \textbf{0.8205}$_{\pm0.0136}$ \\ CricketY & 0.7504$_{\pm0.0065}$ & 0.7043$_{\pm0.003}$ & 0.7812$_{\pm0.003}$ & \textbf{0.8214}$_{\pm0.0039}$ \\ CricketZ & 0.7906$_{\pm0.0039}$ & 0.7333$_{\pm0.0077}$ & 0.7829$_{\pm0.0082}$ & \textbf{0.8462}$_{\pm0.0092}$ \\ Crop & 0.6756$_{\pm0.0018}$ & 0.6738$_{\pm0.0004}$ & 0.7558$_{\pm0.0068}$ & \textbf{0.7681}$_{\pm0.0008}$ \\ DiatomSizeReduction & 0.8845$_{\pm0.0019}$ & 0.8889$_{\pm0.017}$ & 0.8878$_{\pm0.0075}$ & \textbf{0.9314}$_{\pm0.0033}$ \\ DistalPhalanxOutlineAgeGroup & 0.789$_{\pm0.015}$ & \textbf{0.8106}$_{\pm0.0042}$ & 0.6451$_{\pm0.0324}$ & 0.7314$_{\pm0.011}$ \\ DistalPhalanxOutlineCorrect & 0.75$_{\pm0.0126}$ & 0.7621$_{\pm0.0042}$ & \textbf{0.7645}$_{\pm0.0072}$ & 0.7415$_{\pm0.0091}$ \\ DistalPhalanxTW & \textbf{0.6859}$_{\pm0.011}$ & 0.6595$_{\pm0.0083}$ & 0.6619$_{\pm0.0125}$ & 0.6619$_{\pm0.0072}$ \\ DodgerLoopDay & 0.55$_{\pm0.0217}$ & 0.5292$_{\pm0.026}$ & 0.55$_{\pm0.0375}$ & \textbf{0.6125}$_{\pm0.025}$ \\ DodgerLoopGame & 0.7585$_{\pm0.0221}$ & 0.7126$_{\pm0.0084}$ & 0.7633$_{\pm0.0167}$ & \textbf{0.9324}$_{\pm0.0254}$ \\ DodgerLoopWeekend & 0.9517$_{\pm0.0084}$ & 0.9758$_{\pm0.0111}$ & \textbf{0.9831}$_{\pm0.0042}$ & \textbf{0.9831}$_{\pm0.0042}$ \\ ECG200 & 0.82$_{\pm0.01}$ & 0.8033$_{\pm0.0252}$ & 0.8167$_{\pm0.0115}$ & \textbf{0.9067}$_{\pm0.0153}$ \\ ECG5000 & 0.9211$_{\pm0.001}$ & 0.9185$_{\pm0.0038}$ & 0.9301$_{\pm0.0039}$ & \textbf{0.9402}$_{\pm0.0006}$ \\ ECGFiveDays & 0.909$_{\pm0.0218}$ & 0.8668$_{\pm0.0401}$ & 0.7093$_{\pm0.0697}$ & \textbf{0.9512}$_{\pm0.0247}$ \\ EOGHorizontalSignal & 0.5875$_{\pm0.0089}$ & 0.5691$_{\pm0.0083}$ & 0.6308$_{\pm0.0142}$ & \textbf{0.6722}$_{\pm0.0097}$ \\ EOGVerticalSignal & 0.4751$_{\pm0.0}$ & 0.4797$_{\pm0.0131}$ & 0.5037$_{\pm0.0162}$ & \textbf{0.5617}$_{\pm0.018}$ \\ Earthquakes & 0.7482$_{\pm0.0}$ & \textbf{0.7506}$_{\pm0.0042}$ & 0.7482$_{\pm0.0216}$ & 0.7266$_{\pm0.0144}$ \\ ElectricDevices & 0.7226$_{\pm0.0026}$ & 0.7039$_{\pm0.0008}$ & 0.7112$_{\pm0.0091}$ & \textbf{0.7505}$_{\pm0.0068}$ \\ EthanolLevel & 0.2993$_{\pm0.011}$ & 0.3173$_{\pm0.0117}$ & 0.6087$_{\pm0.0463}$ & \textbf{0.6773}$_{\pm0.1125}$ \\ FaceAll & 0.7815$_{\pm0.0074}$ & 0.8053$_{\pm0.0062}$ & \textbf{0.9469}$_{\pm0.0042}$ & 0.8544$_{\pm0.0345}$ \\ FaceFour & 0.9508$_{\pm0.0066}$ & 0.947$_{\pm0.0131}$ & 0.7879$_{\pm0.0237}$ & \textbf{0.9697}$_{\pm0.0131}$ \\ FacesUCR & 0.8354$_{\pm0.0054}$ & 0.8434$_{\pm0.0101}$ & 0.88$_{\pm0.0042}$ & \textbf{0.9411}$_{\pm0.0055}$ \\ FiftyWords & 0.6462$_{\pm0.0096}$ & 0.7026$_{\pm0.0178}$ & 0.8088$_{\pm0.0}$ & \textbf{0.8571}$_{\pm0.0022}$ \\ Fish & 0.9333$_{\pm0.0066}$ & 0.9333$_{\pm0.0033}$ & 0.9143$_{\pm0.0151}$ & \textbf{0.9714}$_{\pm0.0057}$ \\ FordA & 0.8581$_{\pm0.0048}$ & 0.8869$_{\pm0.0016}$ & 0.9235$_{\pm0.0015}$ & \textbf{0.9417}$_{\pm0.002}$ \\ FordB & 0.7305$_{\pm0.0031}$ & 0.765$_{\pm0.0026}$ & 0.7807$_{\pm0.0096}$ & \textbf{0.8329}$_{\pm0.0068}$ \\ FreezerRegularTrain & 0.9374$_{\pm0.0043}$ & 0.8767$_{\pm0.0088}$ & 0.9927$_{\pm0.0026}$ & \textbf{0.9964}$_{\pm0.0015}$ \\ FreezerSmallTrain & 0.7942$_{\pm0.0059}$ & 0.8085$_{\pm0.0041}$ & 0.8469$_{\pm0.0886}$ & \textbf{0.9602}$_{\pm0.0224}$ \\ Fungi & 0.8262$_{\pm0.0164}$ & 0.8477$_{\pm0.0224}$ & 0.9301$_{\pm0.0108}$ & \textbf{0.9839}$_{\pm0.0142}$ \\ GestureMidAirD1 & 0.659$_{\pm0.0044}$ & 0.6026$_{\pm0.0044}$ & 0.6872$_{\pm0.016}$ & \textbf{0.7744}$_{\pm0.0044}$ \\ GestureMidAirD2 & 0.6154$_{\pm0.0077}$ & 0.5256$_{\pm0.016}$ & 0.6487$_{\pm0.0118}$ & \textbf{0.6949}$_{\pm0.0194}$ \\ GestureMidAirD3 & 0.3282$_{\pm0.016}$ & 0.3128$_{\pm0.0291}$ & 0.3897$_{\pm0.032}$ & \textbf{0.459}$_{\pm0.0235}$ \\ GesturePebbleZ1 & 0.9283$_{\pm0.0034}$ & 0.9186$_{\pm0.0116}$ & 0.907$_{\pm0.0101}$ & \textbf{0.9341}$_{\pm0.0089}$ \\ GesturePebbleZ2 & 0.9219$_{\pm0.0256}$ & \textbf{0.9451}$_{\pm0.0097}$ & 0.8586$_{\pm0.0256}$ & 0.8481$_{\pm0.0167}$ \\ GunPoint & 0.98$_{\pm0.0067}$ & 0.9822$_{\pm0.0038}$ & 0.9733$_{\pm0.0176}$ & \textbf{0.9978}$_{\pm0.0038}$ \\ GunPointAgeSpan & 0.9905$_{\pm0.0}$ & 0.9821$_{\pm0.0037}$ & 0.9905$_{\pm0.0032}$ & \textbf{0.9968}$_{\pm0.0}$ \\ GunPointMaleVersusFemale & 0.9958$_{\pm0.0018}$ & 0.9863$_{\pm0.0048}$ & 0.9926$_{\pm0.0018}$ & \textbf{0.9979}$_{\pm0.0037}$ \\ GunPointOldVersusYoung & \textbf{0.9968}$_{\pm0.0}$ & \textbf{0.9968}$_{\pm0.0}$ & 0.9926$_{\pm0.0018}$ & \textbf{0.9968}$_{\pm0.0}$ \\ Ham & 0.673$_{\pm0.0145}$ & 0.6857$_{\pm0.0252}$ & \textbf{0.727}$_{\pm0.0145}$ & 0.6762$_{\pm0.0252}$ \\ HandOutlines & 0.9162$_{\pm0.0072}$ & 0.8928$_{\pm0.0068}$ & 0.9279$_{\pm0.0095}$ & \textbf{0.9459}$_{\pm0.0027}$ \\ Haptics & 0.4968$_{\pm0.0032}$ & 0.4665$_{\pm0.0114}$ & 0.4708$_{\pm0.0086}$ & \textbf{0.5487}$_{\pm0.0225}$ \\ Herring & 0.6667$_{\pm0.0239}$ & \textbf{0.7031}$_{\pm0.0156}$ & 0.5365$_{\pm0.0502}$ & 0.6042$_{\pm0.0325}$ \\ HouseTwenty & 0.9412$_{\pm0.0}$ & 0.9496$_{\pm0.0084}$ & 0.9132$_{\pm0.0049}$ & \textbf{0.9748}$_{\pm0.0084}$ \\ InlineSkate & 0.363$_{\pm0.0136}$ & 0.3164$_{\pm0.0119}$ & 0.383$_{\pm0.0282}$ & \textbf{0.4388}$_{\pm0.0128}$ \\ InsectEPGRegularTrain & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.9893$_{\pm0.0046}$ & \textbf{1.0}$_{\pm0.0}$ \\ InsectEPGSmallTrain & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.9424$_{\pm0.0221}$ & \textbf{1.0}$_{\pm0.0}$ \\ InsectWingbeatSound & 0.519$_{\pm0.0044}$ & 0.4785$_{\pm0.0092}$ & 0.5966$_{\pm0.0146}$ & \textbf{0.6093}$_{\pm0.0148}$ \\ \midrule Blink & \textbf{1.0}$_{\pm0.0}$ & 0.9963$_{\pm0.0046}$ & 0.983$_{\pm0.0051}$ & \textbf{1.0}$_{\pm0.0}$ \\ MotionSenseHAR & \textbf{1.0}$_{\pm0.0}$ & 0.9962$_{\pm0.0}$ & 0.9874$_{\pm0.0022}$ & \textbf{1.0}$_{\pm0.0}$ \\ SharePriceIncrease & \textbf{0.6874}$_{\pm0.0}$ & 0.6718$_{\pm0.001}$ & 0.617$_{\pm0.0112}$ & 0.6342$_{\pm0.0119}$ \\ \midrule ArticularyWordRecognition & 0.9933$_{\pm0.0033}$ & 0.9933$_{\pm0.0}$ & \textbf{0.9967}$_{\pm0.0}$ & 0.9933$_{\pm0.0}$ \\ BasicMotions & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ CharacterTrajectories & 0.9396$_{\pm0.0008}$ & 0.9673$_{\pm0.0021}$ & 0.9789$_{\pm0.0028}$ & \textbf{0.9912}$_{\pm0.0011}$ \\ Cricket & \textbf{1.0}$_{\pm0.0}$ & 0.9861$_{\pm0.0}$ & 0.9861$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ ERing & 0.9407$_{\pm0.0098}$ & 0.9296$_{\pm0.0192}$ & 0.9432$_{\pm0.0077}$ & \textbf{0.9901}$_{\pm0.0043}$ \\ EigenWorms & 0.7532$_{\pm0.0159}$ & 0.7379$_{\pm0.0233}$ & 0.6743$_{\pm0.0192}$ & \textbf{0.8219}$_{\pm0.0044}$ \\ Epilepsy & 0.9952$_{\pm0.0042}$ & 0.9928$_{\pm0.0}$ & 0.9879$_{\pm0.0111}$ & \textbf{1.0}$_{\pm0.0}$ \\ EthanolConcentration & 0.27$_{\pm0.0101}$ & 0.2928$_{\pm0.0132}$ & 0.2852$_{\pm0.0201}$ & \textbf{0.3942}$_{\pm0.0295}$ \\ HandMovementDirection & 0.2117$_{\pm0.0413}$ & 0.2477$_{\pm0.0281}$ & \textbf{0.4234}$_{\pm0.0609}$ & 0.3333$_{\pm0.0475}$ \\ \bottomrule \end{tabular} }`{=latex} ```{=latex} \hfill ``` `\scalebox{\scalefactorft}{ \begin{tabular}{l|llll} \toprule & RF & Head & Scratch & Full \\ \midrule ItalyPowerDemand & 0.9077$_{\pm0.0035}$ & 0.9051$_{\pm0.0048}$ & \textbf{0.9644}$_{\pm0.0034}$ & 0.953$_{\pm0.0006}$ \\ LargeKitchenAppliances & 0.7804$_{\pm0.0041}$ & 0.784$_{\pm0.008}$ & \textbf{0.8676}$_{\pm0.0147}$ & 0.8542$_{\pm0.0094}$ \\ Lightning2 & 0.8033$_{\pm0.0}$ & 0.8251$_{\pm0.0189}$ & \textbf{0.8525}$_{\pm0.0328}$ & 0.8197$_{\pm0.0}$ \\ Lightning7 & 0.7763$_{\pm0.0285}$ & 0.7671$_{\pm0.0137}$ & 0.6986$_{\pm0.0137}$ & \textbf{0.8219}$_{\pm0.0137}$ \\ Mallat & 0.8903$_{\pm0.0137}$ & 0.7765$_{\pm0.0019}$ & 0.8691$_{\pm0.0111}$ & \textbf{0.9389}$_{\pm0.0146}$ \\ Meat & 0.9389$_{\pm0.0096}$ & 0.8889$_{\pm0.0255}$ & 0.8167$_{\pm0.0928}$ & \textbf{0.9444}$_{\pm0.0385}$ \\ MedicalImages & 0.7079$_{\pm0.0035}$ & 0.6939$_{\pm0.0142}$ & 0.7412$_{\pm0.0102}$ & \textbf{0.8018}$_{\pm0.0154}$ \\ MelbournePedestrian & 0.9016$_{\pm0.0031}$ & 0.8922$_{\pm0.0033}$ & 0.9576$_{\pm0.0037}$ & \textbf{0.9703}$_{\pm0.0005}$ \\ MiddlePhalanxOutlineAgeGroup & 0.5801$_{\pm0.0099}$ & \textbf{0.5823}$_{\pm0.0228}$ & 0.5043$_{\pm0.0293}$ & 0.5325$_{\pm0.0298}$ \\ MiddlePhalanxOutlineCorrect & 0.8099$_{\pm0.0099}$ & 0.8007$_{\pm0.0124}$ & 0.7927$_{\pm0.0086}$ & \textbf{0.8522}$_{\pm0.0091}$ \\ MiddlePhalanxTW & \textbf{0.5368}$_{\pm0.0099}$ & 0.5087$_{\pm0.0163}$ & 0.5152$_{\pm0.0262}$ & 0.5087$_{\pm0.0187}$ \\ MixedShapesRegularTrain & 0.943$_{\pm0.0044}$ & 0.9387$_{\pm0.0037}$ & 0.9555$_{\pm0.0043}$ & \textbf{0.9803}$_{\pm0.0006}$ \\ MixedShapesSmallTrain & 0.8961$_{\pm0.0004}$ & 0.8936$_{\pm0.0054}$ & 0.8843$_{\pm0.0116}$ & \textbf{0.9567}$_{\pm0.0041}$ \\ MoteStrain & 0.9137$_{\pm0.0136}$ & 0.8988$_{\pm0.0112}$ & 0.9087$_{\pm0.0095}$ & \textbf{0.9241}$_{\pm0.0042}$ \\ NonInvasiveFetalECGThorax1 & 0.6222$_{\pm0.0037}$ & 0.6712$_{\pm0.0018}$ & 0.9035$_{\pm0.0016}$ & \textbf{0.936}$_{\pm0.0056}$ \\ NonInvasiveFetalECGThorax2 & 0.6872$_{\pm0.0025}$ & 0.712$_{\pm0.004}$ & 0.9152$_{\pm0.0013}$ & \textbf{0.9445}$_{\pm0.0018}$ \\ OSULeaf & 0.8747$_{\pm0.0104}$ & 0.8719$_{\pm0.0109}$ & 0.8609$_{\pm0.0104}$ & \textbf{0.9573}$_{\pm0.0086}$ \\ OliveOil & \textbf{0.9333}$_{\pm0.0}$ & 0.4$_{\pm0.0}$ & 0.6889$_{\pm0.0192}$ & 0.9222$_{\pm0.0509}$ \\ PLAID & 0.8181$_{\pm0.0047}$ & 0.545$_{\pm0.0047}$ & 0.897$_{\pm0.0084}$ & \textbf{0.9199}$_{\pm0.0032}$ \\ PhalangesOutlinesCorrect & 0.7786$_{\pm0.0042}$ & 0.7599$_{\pm0.0051}$ & 0.8139$_{\pm0.0029}$ & \textbf{0.8256}$_{\pm0.0125}$ \\ Phoneme & 0.323$_{\pm0.0034}$ & 0.3224$_{\pm0.0008}$ & 0.2683$_{\pm0.0085}$ & \textbf{0.3434}$_{\pm0.0069}$ \\ PickupGestureWiimoteZ & 0.74$_{\pm0.02}$ & 0.7733$_{\pm0.0611}$ & 0.6667$_{\pm0.0231}$ & \textbf{0.7933}$_{\pm0.0306}$ \\ PigAirwayPressure & 0.484$_{\pm0.0147}$ & 0.4455$_{\pm0.0169}$ & 0.1843$_{\pm0.0182}$ & \textbf{0.5176}$_{\pm0.0194}$ \\ PigArtPressure & \textbf{0.9103}$_{\pm0.0028}$ & 0.8221$_{\pm0.0173}$ & 0.6571$_{\pm0.0028}$ & 0.9087$_{\pm0.0127}$ \\ PigCVP & 0.7837$_{\pm0.0127}$ & 0.7724$_{\pm0.0073}$ & 0.4631$_{\pm0.01}$ & \textbf{0.8894}$_{\pm0.0254}$ \\ Plane & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ PowerCons & 0.9093$_{\pm0.0032}$ & 0.8852$_{\pm0.0274}$ & \textbf{0.9926}$_{\pm0.0032}$ & 0.9889$_{\pm0.0}$ \\ ProximalPhalanxOutlineAgeGroup & 0.8537$_{\pm0.0049}$ & \textbf{0.8732}$_{\pm0.0}$ & 0.8276$_{\pm0.0241}$ & 0.8244$_{\pm0.0049}$ \\ ProximalPhalanxOutlineCorrect & 0.8202$_{\pm0.0086}$ & 0.8477$_{\pm0.0105}$ & \textbf{0.8923}$_{\pm0.0189}$ & 0.8889$_{\pm0.013}$ \\ ProximalPhalanxTW & 0.7691$_{\pm0.0028}$ & 0.7675$_{\pm0.0123}$ & \textbf{0.7789}$_{\pm0.0171}$ & 0.761$_{\pm0.0176}$ \\ RefrigerationDevices & \textbf{0.504}$_{\pm0.0122}$ & 0.4978$_{\pm0.0015}$ & 0.4907$_{\pm0.008}$ & 0.4889$_{\pm0.0056}$ \\ Rock & 0.7133$_{\pm0.0115}$ & 0.7667$_{\pm0.0231}$ & 0.7867$_{\pm0.0306}$ & \textbf{0.8533}$_{\pm0.0115}$ \\ ScreenType & 0.4649$_{\pm0.0134}$ & 0.4507$_{\pm0.0092}$ & 0.4711$_{\pm0.0126}$ & \textbf{0.5058}$_{\pm0.0189}$ \\ SemgHandGenderCh2 & 0.9189$_{\pm0.0086}$ & 0.8561$_{\pm0.0019}$ & 0.9417$_{\pm0.0133}$ & \textbf{0.9461}$_{\pm0.0212}$ \\ SemgHandMovementCh2 & 0.797$_{\pm0.0056}$ & 0.5622$_{\pm0.0174}$ & 0.8267$_{\pm0.0212}$ & \textbf{0.8556}$_{\pm0.025}$ \\ SemgHandSubjectCh2 & 0.8622$_{\pm0.0135}$ & 0.7763$_{\pm0.0064}$ & 0.9193$_{\pm0.0168}$ & \textbf{0.9415}$_{\pm0.021}$ \\ ShakeGestureWiimoteZ & 0.8867$_{\pm0.0115}$ & 0.8933$_{\pm0.0231}$ & 0.84$_{\pm0.02}$ & \textbf{0.9333}$_{\pm0.0115}$ \\ ShapeletSim & 0.9278$_{\pm0.0111}$ & 0.913$_{\pm0.0064}$ & 0.9815$_{\pm0.017}$ & \textbf{1.0}$_{\pm0.0}$ \\ ShapesAll & 0.8194$_{\pm0.0054}$ & 0.8489$_{\pm0.0051}$ & 0.8739$_{\pm0.01}$ & \textbf{0.9039}$_{\pm0.0035}$ \\ SmallKitchenAppliances & 0.8089$_{\pm0.0081}$ & \textbf{0.8293}$_{\pm0.0}$ & 0.7902$_{\pm0.0081}$ & 0.7893$_{\pm0.0071}$ \\ SmoothSubspace & 0.9067$_{\pm0.0115}$ & 0.9378$_{\pm0.0168}$ & 0.98$_{\pm0.0}$ & \textbf{0.9933}$_{\pm0.0}$ \\ SonyAIBORobotSurface1 & 0.787$_{\pm0.0294}$ & 0.8236$_{\pm0.0292}$ & \textbf{0.8264}$_{\pm0.0217}$ & 0.8103$_{\pm0.0202}$ \\ SonyAIBORobotSurface2 & 0.8552$_{\pm0.0168}$ & 0.8821$_{\pm0.0063}$ & 0.8356$_{\pm0.0221}$ & \textbf{0.9307}$_{\pm0.0028}$ \\ StarLightCurves & 0.9759$_{\pm0.0005}$ & \textbf{0.9785}$_{\pm0.0007}$ & 0.972$_{\pm0.002}$ & 0.9768$_{\pm0.0011}$ \\ Strawberry & 0.9514$_{\pm0.0027}$ & 0.9108$_{\pm0.0047}$ & \textbf{0.9712}$_{\pm0.0095}$ & 0.9667$_{\pm0.0041}$ \\ SwedishLeaf & 0.9275$_{\pm0.0009}$ & 0.9253$_{\pm0.0024}$ & 0.9419$_{\pm0.0033}$ & \textbf{0.9701}$_{\pm0.0072}$ \\ Symbols & 0.9698$_{\pm0.0056}$ & 0.9668$_{\pm0.0036}$ & 0.9199$_{\pm0.018}$ & \textbf{0.9873}$_{\pm0.0061}$ \\ SyntheticControl & 0.9767$_{\pm0.0067}$ & 0.9844$_{\pm0.0051}$ & 0.9667$_{\pm0.0067}$ & \textbf{0.9944}$_{\pm0.0038}$ \\ ToeSegmentation1 & 0.9635$_{\pm0.0067}$ & 0.9532$_{\pm0.0127}$ & 0.8684$_{\pm0.0219}$ & \textbf{0.9737}$_{\pm0.0044}$ \\ ToeSegmentation2 & 0.9282$_{\pm0.0044}$ & \textbf{0.9564}$_{\pm0.0089}$ & 0.8692$_{\pm0.0231}$ & 0.9256$_{\pm0.0044}$ \\ Trace & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ TwoLeadECG & 0.9962$_{\pm0.0005}$ & \textbf{0.998}$_{\pm0.0005}$ & 0.9236$_{\pm0.0279}$ & 0.9974$_{\pm0.0015}$ \\ TwoPatterns & 0.8802$_{\pm0.0079}$ & 0.892$_{\pm0.0021}$ & 0.9997$_{\pm0.0004}$ & \textbf{1.0}$_{\pm0.0}$ \\ UMD & 0.9722$_{\pm0.0069}$ & 0.9792$_{\pm0.0}$ & 0.9769$_{\pm0.016}$ & \textbf{0.9931}$_{\pm0.0}$ \\ UWaveGestureLibraryAll & 0.8458$_{\pm0.0057}$ & 0.8733$_{\pm0.0014}$ & 0.9718$_{\pm0.0005}$ & \textbf{0.9746}$_{\pm0.001}$ \\ UWaveGestureLibraryX & 0.7696$_{\pm0.0028}$ & 0.7911$_{\pm0.0017}$ & 0.8368$_{\pm0.0058}$ & \textbf{0.8714}$_{\pm0.0021}$ \\ UWaveGestureLibraryY & 0.6874$_{\pm0.0022}$ & 0.7061$_{\pm0.0024}$ & 0.7742$_{\pm0.0027}$ & \textbf{0.807}$_{\pm0.0011}$ \\ UWaveGestureLibraryZ & 0.7324$_{\pm0.0036}$ & 0.7581$_{\pm0.002}$ & 0.7841$_{\pm0.0011}$ & \textbf{0.8248}$_{\pm0.0011}$ \\ Wafer & 0.9903$_{\pm0.0003}$ & 0.9839$_{\pm0.0069}$ & 0.9948$_{\pm0.0008}$ & \textbf{0.9988}$_{\pm0.0005}$ \\ Wine & \textbf{0.7901}$_{\pm0.0107}$ & 0.5926$_{\pm0.0321}$ & 0.6296$_{\pm0.0185}$ & 0.6914$_{\pm0.0385}$ \\ WordSynonyms & 0.5502$_{\pm0.0081}$ & 0.4901$_{\pm0.0128}$ & 0.6886$_{\pm0.0059}$ & \textbf{0.7571}$_{\pm0.0054}$ \\ Worms & 0.6537$_{\pm0.027}$ & 0.7186$_{\pm0.015}$ & 0.7143$_{\pm0.026}$ & \textbf{0.7489}$_{\pm0.0075}$ \\ WormsTwoClass & \textbf{0.8139}$_{\pm0.0198}$ & \textbf{0.8139}$_{\pm0.0198}$ & 0.7879$_{\pm0.0327}$ & 0.7965$_{\pm0.015}$ \\ Yoga & 0.815$_{\pm0.0012}$ & 0.7452$_{\pm0.0013}$ & 0.8217$_{\pm0.0099}$ & \textbf{0.9061}$_{\pm0.0021}$ \\ \midrule Handwriting & 0.3392$_{\pm0.01}$ & 0.4271$_{\pm0.0113}$ & 0.3294$_{\pm0.0116}$ & \textbf{0.4773}$_{\pm0.0119}$ \\ JapaneseVowels & 0.9631$_{\pm0.0056}$ & 0.9649$_{\pm0.0}$ & 0.9315$_{\pm0.0145}$ & \textbf{0.9757}$_{\pm0.0027}$ \\ LSST & 0.6054$_{\pm0.0021}$ & 0.6459$_{\pm0.0006}$ & 0.666$_{\pm0.0111}$ & \textbf{0.6741}$_{\pm0.0021}$ \\ Libras & 0.8981$_{\pm0.0085}$ & 0.8963$_{\pm0.014}$ & 0.9185$_{\pm0.0064}$ & \textbf{0.9278}$_{\pm0.0111}$ \\ NATOPS & 0.9074$_{\pm0.0128}$ & \textbf{0.9259}$_{\pm0.0116}$ & 0.8079$_{\pm0.0189}$ & \textbf{0.9259}$_{\pm0.0085}$ \\ PhonemeSpectra & 0.2735$_{\pm0.0078}$ & 0.2934$_{\pm0.0002}$ & 0.2082$_{\pm0.0067}$ & \textbf{0.3106}$_{\pm0.0081}$ \\ RacketSports & 0.9232$_{\pm0.0038}$ & 0.9276$_{\pm0.0066}$ & 0.8487$_{\pm0.0066}$ & \textbf{0.9298}$_{\pm0.01}$ \\ SelfRegulationSCP1 & 0.8043$_{\pm0.011}$ & 0.8248$_{\pm0.0052}$ & \textbf{0.8487}$_{\pm0.0071}$ & 0.843$_{\pm0.0034}$ \\ SelfRegulationSCP2 & 0.4759$_{\pm0.0452}$ & 0.487$_{\pm0.0085}$ & 0.487$_{\pm0.014}$ & \textbf{0.4889}$_{\pm0.0309}$ \\ SpokenArabicDigits & 0.8392$_{\pm0.0046}$ & 0.9291$_{\pm0.0016}$ & 0.982$_{\pm0.0027}$ & \textbf{0.9861}$_{\pm0.0009}$ \\ UWaveGestureLibrary & 0.8135$_{\pm0.01}$ & 0.8594$_{\pm0.0031}$ & 0.8833$_{\pm0.0208}$ & \textbf{0.9385}$_{\pm0.011}$ \\ \midrule \textit{\textbf{Best Count}} & 17 & 19 & 18 & \textbf{118} \\ \bottomrule \end{tabular} }`{=latex} `\scalebox{\scalefactorft}{ \begin{tabular}{l|llll} \toprule & RF & Head & Scratch & Full \\ \midrule ACSF1 & 0.6133$_{\pm0.0208}$ & 0.62$_{\pm0.0265}$ & 0.6433$_{\pm0.0451}$ & \textbf{0.7467}$_{\pm0.0416}$ \\ Adiac & 0.7332$_{\pm0.0039}$ & 0.5857$_{\pm0.0051}$ & 0.7613$_{\pm0.0129}$ & \textbf{0.8295}$_{\pm0.0082}$ \\ AllGestureWiimoteX & 0.6705$_{\pm0.0044}$ & 0.551$_{\pm0.0008}$ & 0.6962$_{\pm0.0058}$ & \textbf{0.8029}$_{\pm0.0065}$ \\ AllGestureWiimoteY & 0.6671$_{\pm0.0057}$ & 0.5614$_{\pm0.0052}$ & 0.739$_{\pm0.0008}$ & \textbf{0.8129}$_{\pm0.0089}$ \\ AllGestureWiimoteZ & 0.6695$_{\pm0.0033}$ & 0.5467$_{\pm0.0008}$ & 0.6367$_{\pm0.0246}$ & \textbf{0.7481}$_{\pm0.0095}$ \\ ArrowHead & 0.7105$_{\pm0.0175}$ & 0.7276$_{\pm0.0119}$ & 0.8$_{\pm0.0286}$ & \textbf{0.8229}$_{\pm0.0297}$ \\ BME & 0.9311$_{\pm0.0038}$ & 0.8756$_{\pm0.0139}$ & 0.9933$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ Beef & 0.6556$_{\pm0.0509}$ & 0.6222$_{\pm0.0385}$ & 0.8222$_{\pm0.0694}$ & \textbf{0.8667}$_{\pm0.0}$ \\ BeetleFly & 0.85$_{\pm0.05}$ & \textbf{0.9}$_{\pm0.0}$ & 0.7$_{\pm0.0}$ & \textbf{0.9}$_{\pm0.05}$ \\ BirdChicken & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.9$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ CBF & 0.993$_{\pm0.0013}$ & 0.9941$_{\pm0.0006}$ & 0.9922$_{\pm0.008}$ & \textbf{0.9996}$_{\pm0.0006}$ \\ Car & 0.7722$_{\pm0.0419}$ & 0.7778$_{\pm0.0192}$ & 0.8$_{\pm0.0}$ & \textbf{0.9}$_{\pm0.0289}$ \\ Chinatown & 0.8737$_{\pm0.0168}$ & 0.8785$_{\pm0.0094}$ & 0.964$_{\pm0.0017}$ & \textbf{0.9776}$_{\pm0.0067}$ \\ ChlorineConcentration & 0.6806$_{\pm0.0019}$ & 0.5704$_{\pm0.0011}$ & 0.753$_{\pm0.0056}$ & \textbf{0.7788}$_{\pm0.0072}$ \\ CinCECGTorso & 0.6611$_{\pm0.0036}$ & 0.7179$_{\pm0.0048}$ & 0.7901$_{\pm0.0198}$ & \textbf{0.856}$_{\pm0.0077}$ \\ Coffee & 0.9524$_{\pm0.0206}$ & 0.9643$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ Computers & 0.7373$_{\pm0.0092}$ & 0.728$_{\pm0.0069}$ & 0.7533$_{\pm0.0162}$ & \textbf{0.7773}$_{\pm0.0046}$ \\ CricketX & 0.7368$_{\pm0.0171}$ & 0.6966$_{\pm0.003}$ & 0.7812$_{\pm0.0146}$ & \textbf{0.8282}$_{\pm0.0133}$ \\ CricketY & 0.7504$_{\pm0.0065}$ & 0.7051$_{\pm0.0026}$ & 0.8$_{\pm0.0026}$ & \textbf{0.8385}$_{\pm0.0051}$ \\ CricketZ & 0.7906$_{\pm0.0039}$ & 0.7333$_{\pm0.0077}$ & 0.7991$_{\pm0.0097}$ & \textbf{0.8521}$_{\pm0.0082}$ \\ Crop & 0.6756$_{\pm0.0018}$ & 0.6745$_{\pm0.0001}$ & 0.7566$_{\pm0.0063}$ & \textbf{0.7702}$_{\pm0.0005}$ \\ DiatomSizeReduction & 0.8845$_{\pm0.0019}$ & 0.8911$_{\pm0.0191}$ & 0.9031$_{\pm0.0147}$ & \textbf{0.9346}$_{\pm0.0033}$ \\ DistalPhalanxOutlineAgeGroup & 0.789$_{\pm0.015}$ & \textbf{0.8106}$_{\pm0.0042}$ & 0.7506$_{\pm0.011}$ & 0.7938$_{\pm0.0042}$ \\ DistalPhalanxOutlineCorrect & 0.75$_{\pm0.0126}$ & 0.7729$_{\pm0.0055}$ & \textbf{0.8056}$_{\pm0.0091}$ & 0.7935$_{\pm0.0036}$ \\ DistalPhalanxTW & 0.6859$_{\pm0.011}$ & 0.6763$_{\pm0.0125}$ & 0.7266$_{\pm0.0125}$ & \textbf{0.7338}$_{\pm0.019}$ \\ DodgerLoopDay & 0.55$_{\pm0.0217}$ & 0.5292$_{\pm0.026}$ & 0.6208$_{\pm0.0191}$ & \textbf{0.6375}$_{\pm0.0125}$ \\ DodgerLoopGame & 0.7585$_{\pm0.0221}$ & 0.7222$_{\pm0.0111}$ & 0.843$_{\pm0.0084}$ & \textbf{0.9541}$_{\pm0.0042}$ \\ DodgerLoopWeekend & 0.9517$_{\pm0.0084}$ & 0.9783$_{\pm0.0126}$ & \textbf{0.9855}$_{\pm0.0}$ & \textbf{0.9855}$_{\pm0.0}$ \\ ECG200 & 0.82$_{\pm0.01}$ & 0.8033$_{\pm0.0252}$ & 0.8467$_{\pm0.0115}$ & \textbf{0.9133}$_{\pm0.0115}$ \\ ECG5000 & 0.9211$_{\pm0.001}$ & 0.9185$_{\pm0.0038}$ & 0.9411$_{\pm0.0008}$ & \textbf{0.9459}$_{\pm0.0014}$ \\ ECGFiveDays & 0.909$_{\pm0.0218}$ & 0.8668$_{\pm0.0401}$ & 0.7855$_{\pm0.0169}$ & \textbf{0.9694}$_{\pm0.014}$ \\ EOGHorizontalSignal & 0.5875$_{\pm0.0089}$ & 0.5783$_{\pm0.0064}$ & 0.6703$_{\pm0.0194}$ & \textbf{0.6934}$_{\pm0.012}$ \\ EOGVerticalSignal & 0.4751$_{\pm0.0}$ & 0.4825$_{\pm0.0105}$ & 0.5396$_{\pm0.0058}$ & \textbf{0.5773}$_{\pm0.0073}$ \\ Earthquakes & 0.7482$_{\pm0.0}$ & 0.7506$_{\pm0.0042}$ & \textbf{0.7866}$_{\pm0.015}$ & 0.7602$_{\pm0.015}$ \\ ElectricDevices & 0.7226$_{\pm0.0026}$ & 0.7064$_{\pm0.001}$ & 0.7448$_{\pm0.0044}$ & \textbf{0.7643}$_{\pm0.005}$ \\ EthanolLevel & 0.2993$_{\pm0.011}$ & 0.324$_{\pm0.006}$ & 0.6987$_{\pm0.0194}$ & \textbf{0.7393}$_{\pm0.1052}$ \\ FaceAll & 0.7815$_{\pm0.0074}$ & 0.8053$_{\pm0.0062}$ & \textbf{0.9542}$_{\pm0.0036}$ & 0.8554$_{\pm0.0343}$ \\ FaceFour & 0.9508$_{\pm0.0066}$ & 0.9508$_{\pm0.0174}$ & 0.8485$_{\pm0.0347}$ & \textbf{0.9811}$_{\pm0.0066}$ \\ FacesUCR & 0.8354$_{\pm0.0054}$ & 0.8437$_{\pm0.0104}$ & 0.8816$_{\pm0.0054}$ & \textbf{0.942}$_{\pm0.0061}$ \\ FiftyWords & 0.6462$_{\pm0.0096}$ & 0.7026$_{\pm0.0178}$ & 0.8176$_{\pm0.0038}$ & \textbf{0.863}$_{\pm0.0013}$ \\ Fish & 0.9333$_{\pm0.0066}$ & 0.9333$_{\pm0.0033}$ & 0.939$_{\pm0.0087}$ & \textbf{0.979}$_{\pm0.0087}$ \\ FordA & 0.8581$_{\pm0.0048}$ & 0.8881$_{\pm0.0004}$ & 0.9301$_{\pm0.0031}$ & \textbf{0.9477}$_{\pm0.0013}$ \\ FordB & 0.7305$_{\pm0.0031}$ & 0.7704$_{\pm0.0033}$ & 0.8115$_{\pm0.0036}$ & \textbf{0.8477}$_{\pm0.004}$ \\ FreezerRegularTrain & 0.9374$_{\pm0.0043}$ & 0.8767$_{\pm0.0088}$ & \textbf{0.9971}$_{\pm0.0004}$ & 0.9967$_{\pm0.0014}$ \\ FreezerSmallTrain & 0.7942$_{\pm0.0059}$ & 0.8092$_{\pm0.0041}$ & 0.8601$_{\pm0.073}$ & \textbf{0.9648}$_{\pm0.0219}$ \\ Fungi & 0.8262$_{\pm0.0164}$ & 0.8477$_{\pm0.0224}$ & 0.948$_{\pm0.0164}$ & \textbf{0.9892}$_{\pm0.0108}$ \\ GestureMidAirD1 & 0.659$_{\pm0.0044}$ & 0.6026$_{\pm0.0044}$ & 0.7103$_{\pm0.0089}$ & \textbf{0.7949}$_{\pm0.0044}$ \\ GestureMidAirD2 & 0.6154$_{\pm0.0077}$ & 0.5282$_{\pm0.0118}$ & 0.6974$_{\pm0.0222}$ & \textbf{0.7205}$_{\pm0.0247}$ \\ GestureMidAirD3 & 0.3282$_{\pm0.016}$ & 0.3154$_{\pm0.0266}$ & 0.4359$_{\pm0.016}$ & \textbf{0.4795}$_{\pm0.0178}$ \\ GesturePebbleZ1 & 0.9283$_{\pm0.0034}$ & 0.9186$_{\pm0.0116}$ & 0.9322$_{\pm0.0089}$ & \textbf{0.9574}$_{\pm0.0146}$ \\ GesturePebbleZ2 & 0.9219$_{\pm0.0256}$ & \textbf{0.9515}$_{\pm0.0097}$ & 0.8882$_{\pm0.0193}$ & 0.9451$_{\pm0.0159}$ \\ GunPoint & 0.98$_{\pm0.0067}$ & 0.9822$_{\pm0.0038}$ & 0.9867$_{\pm0.0133}$ & \textbf{0.9978}$_{\pm0.0038}$ \\ GunPointAgeSpan & 0.9905$_{\pm0.0}$ & 0.9821$_{\pm0.0037}$ & 0.9947$_{\pm0.0048}$ & \textbf{0.9989}$_{\pm0.0018}$ \\ GunPointMaleVersusFemale & 0.9958$_{\pm0.0018}$ & 0.9863$_{\pm0.0048}$ & 0.9947$_{\pm0.0018}$ & \textbf{1.0}$_{\pm0.0}$ \\ GunPointOldVersusYoung & 0.9968$_{\pm0.0}$ & 0.9968$_{\pm0.0}$ & 0.9968$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ Ham & 0.673$_{\pm0.0145}$ & 0.7143$_{\pm0.0252}$ & \textbf{0.8159}$_{\pm0.0198}$ & 0.7016$_{\pm0.0198}$ \\ HandOutlines & 0.9162$_{\pm0.0072}$ & 0.8928$_{\pm0.0068}$ & 0.945$_{\pm0.0056}$ & \textbf{0.955}$_{\pm0.0016}$ \\ Haptics & 0.4968$_{\pm0.0032}$ & 0.4697$_{\pm0.0131}$ & 0.5032$_{\pm0.0065}$ & \textbf{0.5747}$_{\pm0.0234}$ \\ Herring & 0.6667$_{\pm0.0239}$ & 0.7188$_{\pm0.0271}$ & 0.6458$_{\pm0.0239}$ & \textbf{0.724}$_{\pm0.009}$ \\ HouseTwenty & 0.9412$_{\pm0.0}$ & 0.9496$_{\pm0.0084}$ & 0.9356$_{\pm0.0049}$ & \textbf{0.9776}$_{\pm0.0097}$ \\ InlineSkate & 0.363$_{\pm0.0136}$ & 0.3182$_{\pm0.0101}$ & 0.403$_{\pm0.0211}$ & \textbf{0.4515}$_{\pm0.0132}$ \\ InsectEPGRegularTrain & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.992$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ InsectEPGSmallTrain & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & 0.9665$_{\pm0.0061}$ & \textbf{1.0}$_{\pm0.0}$ \\ InsectWingbeatSound & 0.519$_{\pm0.0044}$ & 0.4791$_{\pm0.0099}$ & 0.6222$_{\pm0.0066}$ & \textbf{0.6246}$_{\pm0.0068}$ \\ \midrule Blink & \textbf{1.0}$_{\pm0.0}$ & 0.9963$_{\pm0.0046}$ & 0.9896$_{\pm0.0056}$ & \textbf{1.0}$_{\pm0.0}$ \\ MotionSenseHAR & \textbf{1.0}$_{\pm0.0}$ & 0.9962$_{\pm0.0}$ & 0.9887$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ SharePriceIncrease & 0.6874$_{\pm0.0}$ & 0.6746$_{\pm0.0012}$ & \textbf{0.6891}$_{\pm0.0039}$ & 0.6874$_{\pm0.0027}$ \\ \midrule ArticularyWordRecognition & 0.9933$_{\pm0.0033}$ & 0.9933$_{\pm0.0}$ & \textbf{0.9967}$_{\pm0.0}$ & 0.9933$_{\pm0.0}$ \\ BasicMotions & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ CharacterTrajectories & 0.9396$_{\pm0.0008}$ & 0.9673$_{\pm0.0021}$ & 0.9838$_{\pm0.0029}$ & \textbf{0.9928}$_{\pm0.0004}$ \\ Cricket & \textbf{1.0}$_{\pm0.0}$ & 0.9907$_{\pm0.008}$ & 0.9861$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ ERing & 0.9407$_{\pm0.0098}$ & 0.9296$_{\pm0.0192}$ & 0.9432$_{\pm0.0077}$ & \textbf{0.9926}$_{\pm0.0074}$ \\ EigenWorms & 0.7532$_{\pm0.0159}$ & 0.7379$_{\pm0.0233}$ & 0.7023$_{\pm0.0202}$ & \textbf{0.8372}$_{\pm0.0044}$ \\ Epilepsy & 0.9952$_{\pm0.0042}$ & 0.9928$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ EthanolConcentration & 0.27$_{\pm0.0101}$ & 0.3156$_{\pm0.0076}$ & 0.3321$_{\pm0.0154}$ & \textbf{0.4208}$_{\pm0.0195}$ \\ HandMovementDirection & 0.2117$_{\pm0.0413}$ & 0.3829$_{\pm0.0639}$ & \textbf{0.5135}$_{\pm0.0234}$ & 0.4009$_{\pm0.0206}$ \\ \bottomrule \end{tabular} }`{=latex} ```{=latex} \hfill ``` `\scalebox{\scalefactorft}{ \begin{tabular}{l|llll} \toprule & RF & Head & Scratch & Full \\ \midrule ItalyPowerDemand & 0.9077$_{\pm0.0035}$ & 0.9054$_{\pm0.0046}$ & \textbf{0.9725}$_{\pm0.002}$ & 0.9572$_{\pm0.0}$ \\ LargeKitchenAppliances & 0.7804$_{\pm0.0041}$ & 0.7876$_{\pm0.0108}$ & \textbf{0.8853}$_{\pm0.0053}$ & 0.8676$_{\pm0.0094}$ \\ Lightning2 & 0.8033$_{\pm0.0}$ & 0.8251$_{\pm0.0189}$ & \textbf{0.8798}$_{\pm0.025}$ & 0.8361$_{\pm0.0}$ \\ Lightning7 & 0.7763$_{\pm0.0285}$ & 0.7763$_{\pm0.0209}$ & 0.7671$_{\pm0.0137}$ & \textbf{0.8356}$_{\pm0.0137}$ \\ Mallat & 0.8903$_{\pm0.0137}$ & 0.7765$_{\pm0.0019}$ & 0.9529$_{\pm0.0077}$ & \textbf{0.9774}$_{\pm0.0122}$ \\ Meat & 0.9389$_{\pm0.0096}$ & 0.8944$_{\pm0.0255}$ & 0.9222$_{\pm0.0509}$ & \textbf{0.9778}$_{\pm0.0096}$ \\ MedicalImages & 0.7079$_{\pm0.0035}$ & 0.6961$_{\pm0.0139}$ & 0.7605$_{\pm0.0164}$ & \textbf{0.8127}$_{\pm0.0106}$ \\ MelbournePedestrian & 0.9016$_{\pm0.0031}$ & 0.8922$_{\pm0.0033}$ & 0.9615$_{\pm0.0038}$ & \textbf{0.9728}$_{\pm0.0014}$ \\ MiddlePhalanxOutlineAgeGroup & 0.5801$_{\pm0.0099}$ & 0.6126$_{\pm0.0262}$ & \textbf{0.6407}$_{\pm0.0037}$ & 0.6385$_{\pm0.0037}$ \\ MiddlePhalanxOutlineCorrect & 0.8099$_{\pm0.0099}$ & 0.8076$_{\pm0.0119}$ & 0.8328$_{\pm0.0159}$ & \textbf{0.8786}$_{\pm0.0079}$ \\ MiddlePhalanxTW & 0.5368$_{\pm0.0099}$ & 0.5823$_{\pm0.0228}$ & \textbf{0.5974}$_{\pm0.013}$ & 0.5931$_{\pm0.0228}$ \\ MixedShapesRegularTrain & 0.943$_{\pm0.0044}$ & 0.9388$_{\pm0.004}$ & 0.9625$_{\pm0.0014}$ & \textbf{0.9812}$_{\pm0.0005}$ \\ MixedShapesSmallTrain & 0.8961$_{\pm0.0004}$ & 0.8937$_{\pm0.0054}$ & 0.8983$_{\pm0.0114}$ & \textbf{0.9581}$_{\pm0.0046}$ \\ MoteStrain & 0.9137$_{\pm0.0136}$ & 0.9023$_{\pm0.0113}$ & \textbf{0.9358}$_{\pm0.0009}$ & 0.9289$_{\pm0.0085}$ \\ NonInvasiveFetalECGThorax1 & 0.6222$_{\pm0.0037}$ & 0.6733$_{\pm0.0013}$ & 0.9113$_{\pm0.0011}$ & \textbf{0.9411}$_{\pm0.0026}$ \\ NonInvasiveFetalECGThorax2 & 0.6872$_{\pm0.0025}$ & 0.715$_{\pm0.0028}$ & 0.9176$_{\pm0.0013}$ & \textbf{0.9481}$_{\pm0.0027}$ \\ OSULeaf & 0.8747$_{\pm0.0104}$ & 0.8719$_{\pm0.0109}$ & 0.8774$_{\pm0.0063}$ & \textbf{0.9642}$_{\pm0.0167}$ \\ OliveOil & 0.9333$_{\pm0.0}$ & 0.4$_{\pm0.0}$ & 0.8333$_{\pm0.0}$ & \textbf{0.9556}$_{\pm0.0192}$ \\ PLAID & 0.8181$_{\pm0.0047}$ & 0.5469$_{\pm0.0022}$ & 0.9044$_{\pm0.0047}$ & \textbf{0.9243}$_{\pm0.0039}$ \\ PhalangesOutlinesCorrect & 0.7786$_{\pm0.0042}$ & 0.7692$_{\pm0.0012}$ & 0.8322$_{\pm0.0042}$ & \textbf{0.8539}$_{\pm0.0013}$ \\ Phoneme & 0.323$_{\pm0.0034}$ & 0.3231$_{\pm0.002}$ & 0.2795$_{\pm0.0024}$ & \textbf{0.3499}$_{\pm0.0075}$ \\ PickupGestureWiimoteZ & 0.74$_{\pm0.02}$ & 0.7867$_{\pm0.0702}$ & 0.7067$_{\pm0.0306}$ & \textbf{0.8267}$_{\pm0.0306}$ \\ PigAirwayPressure & 0.484$_{\pm0.0147}$ & 0.4519$_{\pm0.0096}$ & 0.2035$_{\pm0.0147}$ & \textbf{0.524}$_{\pm0.0173}$ \\ PigArtPressure & 0.9103$_{\pm0.0028}$ & 0.8221$_{\pm0.0173}$ & 0.6875$_{\pm0.0}$ & \textbf{0.9135}$_{\pm0.0096}$ \\ PigCVP & 0.7837$_{\pm0.0127}$ & 0.7788$_{\pm0.0}$ & 0.4984$_{\pm0.01}$ & \textbf{0.8974}$_{\pm0.0222}$ \\ Plane & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ PowerCons & 0.9093$_{\pm0.0032}$ & 0.8944$_{\pm0.02}$ & \textbf{0.9944}$_{\pm0.0}$ & 0.9889$_{\pm0.0}$ \\ ProximalPhalanxOutlineAgeGroup & 0.8537$_{\pm0.0049}$ & 0.8732$_{\pm0.0}$ & 0.8813$_{\pm0.0028}$ & \textbf{0.8829}$_{\pm0.0084}$ \\ ProximalPhalanxOutlineCorrect & 0.8202$_{\pm0.0086}$ & 0.8499$_{\pm0.0086}$ & 0.9118$_{\pm0.0086}$ & \textbf{0.9244}$_{\pm0.0034}$ \\ ProximalPhalanxTW & 0.7691$_{\pm0.0028}$ & 0.7789$_{\pm0.0056}$ & 0.8179$_{\pm0.0028}$ & \textbf{0.8211}$_{\pm0.0056}$ \\ RefrigerationDevices & 0.504$_{\pm0.0122}$ & 0.5298$_{\pm0.0041}$ & \textbf{0.5538}$_{\pm0.0161}$ & 0.5431$_{\pm0.0041}$ \\ Rock & 0.7133$_{\pm0.0115}$ & 0.7667$_{\pm0.0231}$ & 0.82$_{\pm0.02}$ & \textbf{0.8667}$_{\pm0.0115}$ \\ ScreenType & 0.4649$_{\pm0.0134}$ & 0.4809$_{\pm0.012}$ & 0.5182$_{\pm0.0031}$ & \textbf{0.5511}$_{\pm0.0362}$ \\ SemgHandGenderCh2 & 0.9189$_{\pm0.0086}$ & 0.86$_{\pm0.0033}$ & 0.9494$_{\pm0.0098}$ & \textbf{0.9617}$_{\pm0.0145}$ \\ SemgHandMovementCh2 & 0.797$_{\pm0.0056}$ & 0.563$_{\pm0.0168}$ & 0.8378$_{\pm0.0204}$ & \textbf{0.8674}$_{\pm0.0278}$ \\ SemgHandSubjectCh2 & 0.8622$_{\pm0.0135}$ & 0.7763$_{\pm0.0064}$ & 0.9356$_{\pm0.0102}$ & \textbf{0.9496}$_{\pm0.0181}$ \\ ShakeGestureWiimoteZ & 0.8867$_{\pm0.0115}$ & 0.8933$_{\pm0.0231}$ & 0.9$_{\pm0.0}$ & \textbf{0.9667}$_{\pm0.0115}$ \\ ShapeletSim & 0.9278$_{\pm0.0111}$ & 0.9167$_{\pm0.0111}$ & 0.9815$_{\pm0.017}$ & \textbf{1.0}$_{\pm0.0}$ \\ ShapesAll & 0.8194$_{\pm0.0054}$ & 0.8517$_{\pm0.0076}$ & 0.8817$_{\pm0.0083}$ & \textbf{0.9078}$_{\pm0.001}$ \\ SmallKitchenAppliances & 0.8089$_{\pm0.0081}$ & 0.8302$_{\pm0.0015}$ & 0.8347$_{\pm0.0053}$ & \textbf{0.8471}$_{\pm0.0108}$ \\ SmoothSubspace & 0.9067$_{\pm0.0115}$ & 0.9378$_{\pm0.0168}$ & 0.9844$_{\pm0.0038}$ & \textbf{0.9978}$_{\pm0.0038}$ \\ SonyAIBORobotSurface1 & 0.787$_{\pm0.0294}$ & 0.8247$_{\pm0.03}$ & \textbf{0.9257}$_{\pm0.0069}$ & 0.8763$_{\pm0.0603}$ \\ SonyAIBORobotSurface2 & 0.8552$_{\pm0.0168}$ & 0.8821$_{\pm0.0063}$ & 0.8583$_{\pm0.0371}$ & \textbf{0.9339}$_{\pm0.001}$ \\ StarLightCurves & 0.9759$_{\pm0.0005}$ & 0.9788$_{\pm0.0005}$ & 0.9757$_{\pm0.0015}$ & \textbf{0.9816}$_{\pm0.0006}$ \\ Strawberry & 0.9514$_{\pm0.0027}$ & 0.9153$_{\pm0.0041}$ & \textbf{0.9802}$_{\pm0.0031}$ & 0.9775$_{\pm0.0016}$ \\ SwedishLeaf & 0.9275$_{\pm0.0009}$ & 0.9253$_{\pm0.0024}$ & 0.9456$_{\pm0.0016}$ & \textbf{0.9744}$_{\pm0.0028}$ \\ Symbols & 0.9698$_{\pm0.0056}$ & 0.9668$_{\pm0.0036}$ & 0.9263$_{\pm0.0177}$ & \textbf{0.9893}$_{\pm0.0057}$ \\ SyntheticControl & 0.9767$_{\pm0.0067}$ & 0.9844$_{\pm0.0051}$ & 0.9811$_{\pm0.0051}$ & \textbf{0.9956}$_{\pm0.0019}$ \\ ToeSegmentation1 & 0.9635$_{\pm0.0067}$ & 0.9591$_{\pm0.011}$ & 0.8918$_{\pm0.0198}$ & \textbf{0.9825}$_{\pm0.0044}$ \\ ToeSegmentation2 & 0.9282$_{\pm0.0044}$ & \textbf{0.9564}$_{\pm0.0089}$ & 0.9205$_{\pm0.0118}$ & 0.9462$_{\pm0.0077}$ \\ Trace & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ & \textbf{1.0}$_{\pm0.0}$ \\ TwoLeadECG & 0.9962$_{\pm0.0005}$ & \textbf{0.9985}$_{\pm0.0005}$ & 0.9631$_{\pm0.0066}$ & \textbf{0.9985}$_{\pm0.001}$ \\ TwoPatterns & 0.8802$_{\pm0.0079}$ & 0.892$_{\pm0.0021}$ & 0.9998$_{\pm0.0004}$ & \textbf{1.0}$_{\pm0.0}$ \\ UMD & 0.9722$_{\pm0.0069}$ & 0.9792$_{\pm0.0}$ & 0.9884$_{\pm0.008}$ & \textbf{0.9931}$_{\pm0.0}$ \\ UWaveGestureLibraryAll & 0.8458$_{\pm0.0057}$ & 0.8739$_{\pm0.0014}$ & 0.9737$_{\pm0.0008}$ & \textbf{0.9752}$_{\pm0.0007}$ \\ UWaveGestureLibraryX & 0.7696$_{\pm0.0028}$ & 0.7916$_{\pm0.002}$ & 0.843$_{\pm0.0031}$ & \textbf{0.8728}$_{\pm0.0017}$ \\ UWaveGestureLibraryY & 0.6874$_{\pm0.0022}$ & 0.7065$_{\pm0.0024}$ & 0.7772$_{\pm0.0029}$ & \textbf{0.8117}$_{\pm0.0003}$ \\ UWaveGestureLibraryZ & 0.7324$_{\pm0.0036}$ & 0.7591$_{\pm0.002}$ & 0.7886$_{\pm0.0013}$ & \textbf{0.8288}$_{\pm0.0021}$ \\ Wafer & 0.9903$_{\pm0.0003}$ & 0.984$_{\pm0.0069}$ & 0.9953$_{\pm0.0004}$ & \textbf{0.9988}$_{\pm0.0005}$ \\ Wine & 0.7901$_{\pm0.0107}$ & 0.6049$_{\pm0.0466}$ & 0.7593$_{\pm0.0321}$ & \textbf{0.8148}$_{\pm0.0185}$ \\ WordSynonyms & 0.5502$_{\pm0.0081}$ & 0.4916$_{\pm0.0113}$ & 0.6996$_{\pm0.0063}$ & \textbf{0.7649}$_{\pm0.0027}$ \\ Worms & 0.6537$_{\pm0.027}$ & 0.7186$_{\pm0.015}$ & 0.7662$_{\pm0.013}$ & \textbf{0.7792}$_{\pm0.026}$ \\ WormsTwoClass & 0.8139$_{\pm0.0198}$ & 0.8182$_{\pm0.013}$ & 0.8225$_{\pm0.027}$ & \textbf{0.8312}$_{\pm0.013}$ \\ Yoga & 0.815$_{\pm0.0012}$ & 0.7472$_{\pm0.0012}$ & 0.8308$_{\pm0.0076}$ & \textbf{0.9087}$_{\pm0.0022}$ \\ \midrule Handwriting & 0.3392$_{\pm0.01}$ & 0.4275$_{\pm0.0107}$ & 0.3375$_{\pm0.0129}$ & \textbf{0.482}$_{\pm0.0157}$ \\ JapaneseVowels & 0.9631$_{\pm0.0056}$ & 0.9712$_{\pm0.0016}$ & 0.9374$_{\pm0.0165}$ & \textbf{0.9811}$_{\pm0.0054}$ \\ LSST & 0.6054$_{\pm0.0021}$ & 0.6718$_{\pm0.0017}$ & 0.6944$_{\pm0.0023}$ & \textbf{0.7109}$_{\pm0.0015}$ \\ Libras & 0.8981$_{\pm0.0085}$ & 0.8981$_{\pm0.0116}$ & 0.937$_{\pm0.0085}$ & \textbf{0.9389}$_{\pm0.0}$ \\ NATOPS & 0.9074$_{\pm0.0128}$ & 0.9333$_{\pm0.0056}$ & 0.819$_{\pm0.0115}$ & \textbf{0.937}$_{\pm0.0116}$ \\ PhonemeSpectra & 0.2735$_{\pm0.0078}$ & 0.2965$_{\pm0.0014}$ & 0.2415$_{\pm0.0033}$ & \textbf{0.3421}$_{\pm0.0023}$ \\ RacketSports & 0.9232$_{\pm0.0038}$ & 0.9298$_{\pm0.0076}$ & 0.8684$_{\pm0.0066}$ & \textbf{0.9408}$_{\pm0.0}$ \\ SelfRegulationSCP1 & 0.8043$_{\pm0.011}$ & 0.851$_{\pm0.012}$ & 0.8896$_{\pm0.0052}$ & \textbf{0.9135}$_{\pm0.0071}$ \\ SelfRegulationSCP2 & 0.4759$_{\pm0.0452}$ & 0.5315$_{\pm0.0116}$ & \textbf{0.5611}$_{\pm0.0111}$ & 0.5389$_{\pm0.0096}$ \\ SpokenArabicDigits & 0.8392$_{\pm0.0046}$ & 0.9295$_{\pm0.0016}$ & 0.9828$_{\pm0.0025}$ & \textbf{0.987}$_{\pm0.0009}$ \\ UWaveGestureLibrary & 0.8135$_{\pm0.01}$ & 0.8594$_{\pm0.0031}$ & 0.8958$_{\pm0.0079}$ & \textbf{0.9438}$_{\pm0.0108}$ \\ \midrule \textit{\textbf{Best Count}} & 9 & 11 & 25 & \textbf{128} \\ \bottomrule \end{tabular} }`{=latex} ```{=latex} \centering ``` ::: {#tab:rf-vs-best-adapter} dataset RF Full (Best Result) --------------------------- -------------------------- -------------------------- ArticularyWordRecognition 0.9867$_{\pm0.0076}$ **0.9933**$_{\pm0.0}$ BasicMotions 0.9958$_{\pm0.0102}$ **1.0**$_{\pm0.0}$ CharacterTrajectories 0.9546$_{\pm0.0165}$ **0.9947**$_{\pm0.0004}$ Cricket 0.9884$_{\pm0.0137}$ **1.0**$_{\pm0.0}$ DuckDuckGeese 0.42$_{\pm0.031}$ **0.6**$_{\pm0.02}$ ERing 0.9549$_{\pm0.0174}$ **0.9926**$_{\pm0.0074}$ EigenWorms 0.7443$_{\pm0.0172}$ **0.8601**$_{\pm0.0192}$ Epilepsy 0.9915$_{\pm0.0055}$ **1.0**$_{\pm0.0}$ EthanolConcentration 0.2738$_{\pm0.009}$ **0.4208**$_{\pm0.0195}$ FaceDetection 0.5354$_{\pm0.0119}$ **0.6272**$_{\pm0.013}$ FingerMovements 0.5183$_{\pm0.0354}$ **0.6167**$_{\pm0.0058}$ HandMovementDirection 0.2635$_{\pm0.0627}$ **0.5135**$_{\pm0.027}$ Handwriting 0.2876$_{\pm0.0569}$ **0.5839**$_{\pm0.0283}$ Heartbeat 0.7626$_{\pm0.0408}$ **0.7951**$_{\pm0.0129}$ InsectWingbeatSubset 0.425$_{\pm0.1992}$ **0.591**$_{\pm0.005}$ JapaneseVowels 0.9216$_{\pm0.0459}$ **0.9811**$_{\pm0.0054}$ LSST 0.613$_{\pm0.0084}$ **0.7109**$_{\pm0.0015}$ Libras 0.8741$_{\pm0.0297}$ **0.9704**$_{\pm0.0032}$ MotorImagery 0.5483$_{\pm0.0417}$ **0.6**$_{\pm0.01}$ NATOPS 0.8648$_{\pm0.0498}$ **0.9611**$_{\pm0.0}$ PEMS-SF **0.9923**$_{\pm0.0119}$ 0.9114$_{\pm0.0067}$ PhonemeSpectra 0.2444$_{\pm0.0322}$ **0.3547**$_{\pm0.0047}$ RacketSports 0.8717$_{\pm0.0569}$ **0.9408**$_{\pm0.0}$ SelfRegulationSCP1 0.7793$_{\pm0.0293}$ **0.917**$_{\pm0.0052}$ SelfRegulationSCP2 0.4861$_{\pm0.0331}$ **0.5685**$_{\pm0.0251}$ SpokenArabicDigits 0.8872$_{\pm0.0527}$ **0.9933**$_{\pm0.0014}$ UWaveGestureLibrary 0.8432$_{\pm0.0339}$ **0.9438**$_{\pm0.0108}$ : The performance comparison between Mantis in the zero-shot feature extraction setting and when it is fully fine-tuned with or without an adapter, in average over UEA-27 datasets. In the second case, the best possible performance per dataset is taken. :::