Bibliography on Optical Music Recognition

This paper proposes a system for automatically reading either Japanese or English documents that have complex layout structures that include graphics. First, document image segmentation and character segmentation are carried out using three basic features and the knowledge of document layout rules. Next, multi-font character recognition is performed based on feature vector matching. Recognition experiments with a prototype system for a variety of complex printed documents shows that the proposed system is capable of reading different types of printed documents at an accuracy rate of 94.8–97.2%.

This paper focuses on new models to enjoy music that will be implementable in a near future thanks to 5G technology. In the last two decades, our research mainly focused on the comprehensive description of music information, where multiple aspects are integrated to provide the user with an advanced multi-layer environment to experience music content. In recent times, the advancements in network technologies allowed a web implementation of this approach through W3C-compliant languages. The last obstacle to the use of personal devices is currently posed by the characteristics of mobile networks, concerning bandwidth, reliability, and the density of devices in an area. Designed to meet the requirements of future technological challenges, such as the Internet of Things and self-driving vehicles, the advent of 5G networks will solve these problems, thus paving the way also for new music-oriented applications. The possibilities described in this work range from bringing archive materials and music cultural heritage to a new life to the implementation of immersive environments for live-show remote experience.

Automatic music transcription is considered by many to be a key enabling technology in music signal processing. However, the performance of transcription systems is still significantly below that of a human expert, and accuracies reported in recent years seem to have reached a limit, although the field is still very active. In this paper we analyse limitations of current methods and identify promising directions for future research. Current transcription methods use general purpose models which are unable to capture the rich diversity found in music signals. One way to overcome the limited performance of transcription systems is to tailor algorithms to specific use-cases. Semi-automatic approaches are another way of achieving a more reliable transcription. Also, the wealth of musical scores and corresponding audio data now available are a rich potential source of training data, via forced alignment of audio to scores, but large scale utilisation of such data has yet to be attempted. Other promising approaches include the integration of information from multiple algorithms and different musical aspects.

Long Short-Term Memory (LSTM) networks have yielded excellent results on handwriting recognition. This paper describes an application of bidirectional LSTM networks to the problem of machine-printed Latin and Fraktur recognition. Latin and Fraktur recognition differs significantly from handwriting recognition in both the statistical properties of the data, as well as in the required, much higher levels of accuracy. Applications of LSTM networks to handwriting recognition use two-dimensional recurrent networks, since the exact position and baseline of handwritten characters is variable. In contrast, for printed OCR, we used a one-dimensional recurrent network combined with a novel algorithm for baseline and x-height normalization. A number of databases were used for training and testing, including the UW3 database, artificially generated and degraded Fraktur text and scanned pages from a book digitization project. The LSTM architecture achieved 0.6 error on English text. When the artificially degraded Fraktur data set is divided into training and test sets, the system achieves an error rate of 1.64 of the training set), the system achieves error rates of 0.15 and 1.47 without using any language modelling or any other post-processing techniques.

Data reduction techniques play a key role in instance-based classification to lower the amount of data to be processed. Among the different existing approaches, prototype selection (PS) and prototype generation (PG) are the most representative ones. These two families differ in the way the reduced set is obtained from the initial one: While the former aims at selecting the most representative elements from the set, the latter creates new data out of it. Although PG is considered to delimit more efficiently decision boundaries, the operations required are not so well defined in scenarios involving structural data such as strings, trees, or graphs. This work studies the possibility of using dissimilarity space (DS) methods as an intermediate process for mapping the initial structural representation to a statistical one, thereby allowing the use of PG methods. A comparative experiment over string data is carried out in which our proposal is faced to PS methods on the original space. Results show that the proposed strategy is able to achieve significantly similar results to PS in the initial space, thus standing as a clear alternative to the classic approach, with some additional advantages derived from the DS representation.

Abstract Interactive Pattern Recognition (IPR) is an emergent framework in which the user is involved actively in the recognition process by giving feedback to the system when an error is detected. Although this framework is expected to reduce the number of errors to correct, it may increase the time required to complete the task since the machine needs to recompute its proposal after each interaction. Therefore, a fast computation is required to make the interactive system profitable and user-friendly. This work presents an efficient approach to deal with IPR tasks when data has a sequential nature. Our approach includes some computation at the very beginning of the task but it then achieves a linear complexity after user corrections. We also show how these tasks can be effectively carried out if the solution space is defined with a Regular Language. This fact has indeed proven to be the most relevant factor to improve the efficiency of the approach. Several experiments are carried out in which our proposal is faced against a classical search. Results show a reduction in time in all experiments considered, solving efficiently some complex IPR tasks thanks to our proposals.

Expressive performance is an indispensable part of music making. When playing a piece, expert performers shape various parameters (tempo, timing, dynamics, intonation, articulation, etc.) in ways that are not prescribed by the notated score, in this way producing an expressive rendition that brings out dramatic, affective, and emotional qualities that may engage and affect the listeners. Given the central importance of this skill for many kinds of music, expressive performance has become an important research topic for disciplines like musicology, music psychology, etc. This paper focuses on a specific thread of research: work on computational music performance models. Computational models are attempts at codifying hypotheses about expressive performance in terms of mathematical formulas or computer programs, so that they can be evaluated in systematic and quantitative ways. Such models can serve at least two main purposes: they permit us to systematically study certain hypotheses regarding performance; and they can be used as tools to generate automated or semi-automated performances, in artistic or educational contexts. The present article presents an up-to-date overview of the state of the art in this domain. We explore recent trends in the field, such as a strong focus on data-driven (machine learning); a growing interest in interactive expressive systems, such as conductor simulators and automatic accompaniment systems; and an increased interest in exploring cognitively plausible features and models. We provide an in-depth discussion of several important design choices in such computer models, and discuss a crucial (and still largely unsolved) problem that is hindering systematic progress: the question of how to evaluate such models in scientifically and musically meaningful ways. From all this, we finally derive some research directions that should be pursued with priority, in order to advance the field and our understanding of expressive music performance.

Attention-based encoder-decoder architectures such as Listen, Attend, and Spell (LAS), subsume the acoustic, pronunciation and language model components of a traditional automatic speech recognition (ASR) system into a single neural network. In previous work, we have shown that such architectures are comparable to state-of-the-art ASR systems on dictation tasks, but it was not clear if such architectures would be practical for more challenging tasks such as voice search. In this work, we explore a variety of structural and optimization improvements to our LAS model which significantly improve performance. On the structural side, we show that word piece models can be used instead of graphemes. We also introduce a multi-head attention architecture, which offers improvements over the commonly-used single-head attention. On the optimization side, we explore synchronous training, scheduled sampling, label smoothing, and minimum word error rate optimization, which are all shown to improve accuracy. We present results with a unidirectional LSTM encoder for streaming recognition. On a 12, 500 hour voice search task, we find that the proposed changes improve the WER from 9.2% to 5.6%, while the best conventional system achieves 6.7%; on a dictation task our model achieves a WER of 4.1% compared to 5% for the conventional system.

The different aspects of a process for recognizing symbols in documents are considered and the techniques that have been most commonly used during the last ten years, in the different application fields, are reviewed. Methods used in the representation, description and classification phases are shortly discussed and the main recognition strategies are mentioned. Some of the problems that appear still open are proposed to the attention of the reader.

Advances in the science and technology of Music Information Retrieval (MIR) systems and algorithms are dependent on the development of rigorous measures of accuracy and performance such that meaningful comparisons among current and novel approaches can be made. This is the motivating principle driving the efforts of the International Music Information Retrieval Systems Evaluation Laboratory (IMIRSEL) and the annual Music Information Retrieval Evaluation eXchange (MIREX). Since it started in 2005, MIREX has fostered great advancements not only in many specific areas of MIR, but also in our general understanding of how MIR systems and algorithms are to be evaluated. This chapter outlines some of the major highlights of the past four years of MIREX evaluations, including its organizing principles, the selection of evaluation metrics, and the evolution of evaluation tasks. The chapter concludes with a brief introduction of how MIREX plans to expand into the future using a suite of Web 2.0 technologies to automated MIREX evaluations.

We present a new technique for dealing with broken characters, one of the major challenges in the optical character recognition (OCR) of degraded historical printed documents. A technique based on graph combinatorics is used to rejoin the appropriate connected components. It has been applied to real data with successful results.

In document image analysis, segmentation is the task that identifies the regions of a document. The increasing number of applications of document analysis requires a good knowledge of the available technologies. This survey highlights the variety of the approaches that have been proposed for document image segmentation since 2008. It provides a clear typology of documents and of document image segmentation algorithms. We also discuss the technical limitations of these algorithms, the way they are evaluated and the general trends of the community. © 2016 Elsevier Ltd

Previous works have postulated a similarity between music reading and text reading. Therefore it is interesting to evaluate both of these functions in an alexic subject. The patient investigated is a professional musician who had an ischemic lesion in the left temporoparieto-occipital region. Text reading showed pure alexia in which both the phonological and global routes were damaged. His ability to read correctly via matching tests showed that the word-form system was preserved. The reading of musical scores was damaged too and showed a dissociation between the reading of ideograms and rhythms (preserved) and the reading of notes (impaired). The results of note reading were analogous to those of word reading. Furthermore, the patient could read notes correctly via matching tests. On the basis of these findings, we propose a model of music reading where the reading of notes is based on a representational system analogous to that of words (the so-called internal language) whereas reading of ideograms and rhythms occurs via an internal representation unrelated to linguistic functions.

Learn the nuances of music notation, and create professional looking scores. This reference presents a comprehensive look at contemporary music notation. You will learn the meaning and stylistic practices for many types of notation that are currently in common use, from traditional staffs to lead sheets to guitar tablature. It discusses hundreds of notation symbols, as well as general guidelines for writing music. Berklee College of Music brings together teachers and students from all over the world, and we use notation in a great variety of ways. This book presents our perspectives on notation: what we have found to be the most commonly used practices in today's music industry, and what seems to be serving our community best. It includes a foreword by Matthew Nicholl, who was a long-time chair of Berklee's Contemporary Writing and Production Department. Whether you find yourself in a Nashville recording studio, Hollywood sound stage, grand concert hall, worship choir loft, or elementary school auditorium, this book will help you to create readable, professional, publication-quality notation. Beyond understanding the standard rules and definitions, you will learn to make appropriate choices for your own work, and generally how to achieve clarity and consistency in your notation so that it best serves your music.

Classification of musical segments is an interesting problem. It is a key technology in the development of content-based audio document indexing and retrieval. In this paper, we apply the feature extraction and modeling techniques commonly used in automatic speech recognition to solving the problem of segmentation and instrument identification of musical passages. The correlation among the different components in the feature space and the auto-correlation of each component are analyzed to demonstrate feasibility in musical signal analysis and instrument class modeling. Our experimental results are first evaluated on 3 instrument categories, i.e. vocal music, instrumental music, and their combinations. Furthermore each category is split into two individual cases to give a 6-class problem. Our results show that good performance could be obtained with simple features, such as mel-frequency cepstral coefficients and cepstral coefficients derived from linear prediction signal analysis. Even with a limited amount of training data, we could give an accuracy of 90.60% in the case of three categories. A slightly worse accuracy of 90.38% is obtained when we double the number of categories to six classes.

Historical document collections are a valuable resource for human history. This paper proposes a novel digital image binarization scheme for low quality historical documents allowing further content exploitation in an efficient way. The proposed scheme consists of five distinct steps: a pre-processing procedure using a low-pass Wiener filter, a rough estimation of foreground regions using Niblack's approach, a background surface calculation by interpolating neighboring background intensities, a thresholding by combining the calculated background surface with the original image and finally a post-processing step in order to improve the quality of text regions and preserve stroke connectivity. The proposed methodology works with great success even in cases of historical manuscripts with poor quality, shadows, nonuniform illumination, low contrast, large signal- dependent noise, smear and strain. After testing the proposed method on numerous low quality historical manuscripts, it has turned out that our methodology performs better compared to current state-of-the-art adaptive thresholding techniques.

DIBCO 2009 is the first International Document Image Binarization Contest organized in the context of ICDAR 2009 conference. The general objective of the contest is to identify current advances in document image binarization using established evaluation performance measures. This paper describes the contest details including the evaluation measures used as well as the performance of the 43 submitted methods along with a short description of each method.

Vast collections of documents available in image format need to be indexed for information retrieval purposes. In this framework, word spotting is an alternative solution to optical character recognition (OCR), which is rather inefficient for recognizing text of degraded quality and unknown fonts usually appearing in printed text, or writing style variations in handwritten documents. Over the past decade there has been a growing interest in addressing document indexing using word spotting which is reflected by the continuously increasing number of approaches. However, there exist very few comprehensive studies which analyze the various aspects of a word spotting system. This work aims to review the recent approaches as well as fill the gaps in several topics with respect to the related works. The nature of texts and inherent challenges addressed by word spotting methods are thoroughly examined. After presenting the core steps which compose a word spotting system, we investigate the use of retrieval enhancement techniques based on relevance feedback which improve the retrieved results. Finally, we present the datasets which are widely used for word spotting, we describe the evaluation standards and measures applied for performance assessment and discuss the results achieved by the state of the art.

This paper proposes a Fast Region-based Convolutional Network method (Fast R-CNN) for object detection. Fast R-CNN builds on previous work to efficiently classify object proposals using deep convolutional networks. Compared to previous work, Fast R-CNN employs several innovations to improve training and testing speed while also increasing detection accuracy. Fast R-CNN trains the very deep VGG16 network 9x faster than R-CNN, is 213x faster at test-time, and achieves a higher mAP on PASCAL VOC 2012. Compared to SPPnet, Fast R-CNN trains VGG16 3x faster, tests 10x faster, and is more accurate. Fast R-CNN is implemented in Python and C++ (using Caffe) and is available under the open-source MIT License at https://github.com/rbgirshick/fast-rcnn.

Object detection performance, as measured on the canonical PASCAL VOC Challenge datasets, plateaued in the final years of the competition. The best-performing methods were complex ensemble systems that typically combined multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 50 percent relative to the previous best result on VOC 2012-achieving a mAP of 62.4 percent. Our approach combines two ideas: (1) one can apply high-capacity convolutional networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data are scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, boosts performance significantly. Since we combine region proposals with CNNs, we call the resulting model an R-CNN or Region-based Convolutional Network. Source code for the complete system is available at http://www.cs.berkeley.edu/ rbg/rcnn.

The MusicXML format is designed to be a universal translator for programs that understand common Western musical notation. We have made significant progress towards this goal, with over a dozen programs supporting MusicXML as of June 2003. We describe some of the ways that MusicXML has been used for file interchange, and will demonstrate several scenarios.

Recognizing arbitrary multi-character text in unconstrained natural photographs is a hard problem. In this paper, we address an equally hard sub-problem in this domain viz. recognizing arbitrary multi-digit numbers from Street View imagery. Traditional approaches to solve this problem typically separate out the localization, segmentation, and recognition steps. In this paper we propose a unified approach that integrates these three steps via the use of a deep convolutional neural network that operates directly on the image pixels. We employ the DistBelief implementation of deep neural networks in order to train large, distributed neural networks on high quality images. We find that the performance of this approach increases with the depth of the convolutional network, with the best performance occurring in the deepest architecture we trained, with eleven hidden layers. We evaluate this approach on the publicly available SVHN dataset and achieve over 96% accuracy in recognizing complete street numbers. We show that on a per-digit recognition task, we improve upon the state-of-the-art, achieving 97.84% accuracy. We also evaluate this approach on an even more challenging dataset generated from Street View imagery containing several tens of millions of street number annotations and achieve over 90% accuracy. To further explore the applicability of the proposed system to broader text recognition tasks, we apply it to synthetic distorted text from reCAPTCHA. reCAPTCHA is one of the most secure reverse turing tests that uses distorted text to distinguish humans from bots. We report a 99.8% accuracy on the hardest category of reCAPTCHA. Our evaluations on both tasks indicate that at specific operating thresholds, the performance of the proposed system is comparable to, and in some cases exceeds, that of human operators.

Six types of eye movement were measured and recorded with an SRI Eyetracker: number of progressive and regressive fixations, durations of progressive and regressive fixations and lengths of progressive and regressive saccades. Twenty-four graduate music students were selected as skilled and less- skilled music readers. Eye position was measured every millisecond with a high degree of accuracy. The factorial design was 2 Groups x 4 Melodies x 3 Encounters (including a practice period). Results indicated that patterns of eye movement in the two groups were similar across melodies and encounters, but differed with notational complexity. Eye movement was reduced when performing melodies with more-concentrated visual information than when performing melodies with less- concentrated visual information. The main effect of encounters indicated that music readers used fewer but longer fixations after practicing the melodies. Results suggest that skilled music readers look farther ahead in the notation, and then back to the point of performance, when sightreading.

Temporal and sequential components of the eye movement used by a skilled and a less-skilled sightreader were used to construct six profiles of processing. Each subject read three melodies of varying levels of concentration of visual detail. The profiles indicates the order, duration, and location of each fixation while the subjects sightread the melodies. Results indicate that music readers do not fixate on note stems or the bar lines that connect eighth notes when sightreading. The less-skilled music reader progressed through the melody virtually note-by-note using long fixations, whereas the skilled sightreader directed fixations to all areas of the notation (using more regressions than the less-skilled reader) to perform the music accurately. Results support earlier findings that skilled sightreaders look farther ahead in the notation, then back to the point of performance (Goolsby, 1994), and have a larger perceptual span than less-skilled sightreaders. Findings support Slobodans (1984) contention that music reading (i. e., sightreading) is indeed music perception, because music notation is processed before performance. Support was found for Sloboda's (1977, 1984, 1985, 1988) hypotheses on the effects of physical and structural boundaries on visual musical perception. The profiles indicate a number of differences between music perception from processing visual notation and perception resulting from language reading. These differences include: (1) opposite trends in the control of eye movement (i. e., the better music reader fixates in blank areas of the visual stimuli and not directly on each item of the information that was performed), (2) a perceptual span that is vertical as well as horizontal, (3) more eye movement associated with the better reader, and (4) greater attention used for processing language than for music, although the latter task requires an "exact realization."

Recurrent neural networks (RNNs) are a powerful model for sequential data. End-to-end training methods such as Connectionist Temporal Classification make it possible to train RNNs for sequence labelling problems where the input-output alignment is unknown. The combination of these methods with the Long Short-term Memory RNN architecture has proved particularly fruitful, delivering state-of-the-art results in cursive handwriting recognition. However RNN performance in speech recognition has so far been disappointing, with better results returned by deep feedforward networks. This paper investigates deep recurrent neural networks, which combine the multiple levels of representation that have proved so effective in deep networks with the flexible use of long range context that empowers RNNs. When trained end-to-end with suitable regularisation, we find that deep Long Short-term Memory RNNs achieve a test set error of 17.7% on the TIMIT phoneme recognition benchmark, which to our knowledge is the best recorded score.

Current top performing object detectors employ detection proposals to guide the search for objects, thereby avoiding exhaustive sliding window search across images. Despite the popularity and widespread use of detection proposals, it is unclear which trade-offs are made when using them during object detection. We provide an in-depth analysis of twelve proposal methods along with four baselines regarding proposal repeatability, ground truth annotation recall on PASCAL, ImageNet, and MS COCO, and their impact on DPM, R-CNN, and Fast R-CNN detection performance. Our analysis shows that for object detection improving proposal localisation accuracy is as important as improving recall. We introduce a novel metric, the average recall (AR), which rewards both high recall and good localisation and correlates surprisingly well with detection performance. Our findings show common strengths and weaknesses of existing methods, and provide insights and metrics for selecting and tuning proposal methods.

Information about manuscripts and printed music indexed in RISM (Répertoire International des Sources Musicales), a large, international project that records and describes musical sources, was for decades available solely through book publications, CD-ROMs, or subscription services. Recent initiatives to make the data available on a wider scale have resulted in, most significantly, a freely accessible online database and the availability of its data as open data and linked open data. Previously, the task of increasing the amount of data was primarily carried out by RISM national groups and the Zentralredaktion (Central Office). The current opportunities available by linking to other freely accessible databases and importing data from other resources open new perspectives and prospects. This paper describes the RISM data and their applications for digital libraries and digital musicological projects. We discuss the possibilities and challenges in making available a large and growing quantity of data and how the data have been utilized in external library and musicological projects. Interactive functions in the RISM OPAC are planned for the future, as is closer collaboration with the projects that use RISM data. Ultimately, RISM would like to arrange a “take and give” system in which the RISM data are used in external projects, enhanced by the project participants, and then delivered back to the RISM Zentralredaktion.

Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.

Object detection is one of the important problems for autonomous robots. Faster R-CNN, one of the state-of-the-art object detection methods, approaches real time application; nevertheless, computational time lies borderline of real time application, i.e. 5fps with VGG16 model in K40 GPU system in [1]. Moreover, computation time depends on model and image crop size, but precision is also affected; usually, time and precision have trade-off relation. By adjusting input image size in spite of downgrading performance, computation time meets criteria for one model. Therefore, selection of a model is one of the important problems when faster R-CNN based object detection system for an autonomous robot is constructed. In this paper, we convert several state-of-the-art models from convolution neural network (CNN) for image classification. Then, we compare converted models with several image crop size in terms of computation time and detection precision. We will utilize those comparison data for selecting a proper detection model in case a robot needs to perform an object detection task.

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.

We present a method for detecting objects in images using a single deep neural network. Our approach, named SSD, discretizes the output space of bounding boxes into a set of default boxes over different aspect ratios and scales per feature map location. At prediction time, the network generates scores for the presence of each object category in each default box and produces adjustments to the box to better match the object shape. Additionally, the network combines predictions from multiple feature maps with different resolutions to naturally handle objects of various sizes. SSD is simple relative to methods that require object proposals because it completely eliminates proposal generation and subsequent pixel or feature resampling stages and encapsulates all computation in a single network. This makes SSD easy to train and straightforward to integrate into systems that require a detection component. Experimental results on the PASCAL VOC, COCO, and ILSVRC datasets confirm that SSD has competitive accuracy to methods that utilize an additional object proposal step and is much faster, while providing a unified framework for both training and inference. For 300x300 input, SSD achieves 74.3% mAP on VOC2007 test at 59 FPS on a Nvidia Titan X and for 512x512 input, SSD achieves 76.9% mAP, outperforming a comparable state of the art Faster R-CNN model. Compared to other single stage methods, SSD has much better accuracy even with a smaller input image size. Code is available at https://github.com/weiliu89/caffe/tree/ssd.

Parallel to the very first music expressions as a human activity, developed under some kind of guidelines and precepts, there have arisen relentless efforts and attempts to achieve the long-lasting and stable representation of the musical products being created, in a way that would allow to recreate them as closely as posible to the original event.from the very beginning until the present days, have been many notation systems, formats, and expressions developed with this purpose in mind. With the incursion and implementation of the new digital technologies in the musical activity spheres, a new paradigm emerge in the representation and transmission of the musical work, culminating nowadays with the born of a new music document typology tha may be called digital music score or e-Score. The aim of this paper is lay down and approach to this new type of object for music texts reading, editing and transmission through a quantitive research, being the scope of the study to analyze and understand how electronic music scores (e-Score) are received, used and perceived by music students and teachers in any of the music learning centers in our country. The main purpose is to provide scientific evidence, through statistical methods, of the level of implementation and use of these new support type for music knowledge transmission, as well as the most influential aspects implied in perception of therm amongst musical community in Spain.

Antique printed books constitute a heritage that should be preserved and used. With novel digitising techniques is now possible to have these books stored in digital format and accessible to a wider public. However it remains the problem of how to use them. DEBORA (Digital accEss to BOoks of the RenAissance) is a European project that aims to develop a system to interact with these books through world-wide networks. The main issue is to build a database accessible through client computers. That will require to built accompanying metadata that should characterise different components of the books as illuminated letters, banners, figures and key words in order to simplify and speed up the remote access. To solve these problems, digital image analysis algorithms regarding filtering, segmentation, separation of text from non-text, lines and word segmentation and word recognition were developed. Some novel ideas are presented and illustrated through examples.

Many multimedia music content owners and distributors are converting their archives of music scores from paper into digital images, and to machine readable symbolic notation in order to survive in the business world. Interactive Multimedia Music Technologies discusses relevant state-of-the-art technologies and consists of analysis, knowledge, and application scenarios as surveyed, analyzed, and evaluated by industry professionals. Interactive Multimedia Music Technologies exemplifies the newest functionalities of multimedia interactive music to be used for valorizing cultural heritage, content and archives that are not currently distributed due to lack of safety, suitable coding models, and conversion technologies. Interactive Multimedia Music Technologies explains new and innovative methods of promoting music and products for entertainment, distance teaching, valorizing archives, and commercial and non-commercial purposes, and provides new services for those connected via personal computers, mobile and other devices, for both sighted and print-impaired consumers.

Document image binarization is of great importance in the document image analysis and recognition pipeline since it affects further stages of the recognition process. The evaluation of a binarization method aids in studying its algorithmic behavior, as well as verifying its effectiveness, by providing qualitative and quantitative indication of its performance. This paper addresses a pixel-based binarization evaluation methodology for historical handwritten/machine-printed document images. In the proposed evaluation scheme, the recall and precision evaluation measures are properly modified using a weighting scheme that diminishes any potential evaluation bias. Additional performance metrics of the proposed evaluation scheme consist of the percentage rates of broken and missed text, false alarms, background noise, character enlargement, and merging. Several experiments conducted in comparison with other pixel-based evaluation measures demonstrate the validity of the proposed evaluation scheme.

A Hidden Markov Model (HMM) is a powerful model in describing temporal sequences. The HMM parameters are usually estimated using Baum-Welch algorithm. However, it is well known that the Baum-Welch algorithm tends to arrive at local optimal points. In this report, we investigate the potential of the Particle Swarm Optimisation (PSO) as an alternative method for HMM parameters estimation. The domain in this study is the recognition of handwritten music notations. Three observables: (i) sequence of ink patterns, (ii) stroke information and (iii) spatial information associated with eight musical symbols were recorded. Sixteen HMM models were built from the data. Eight HMM models for eight musical symbols were built from the parameters estimated using the Baum-Welch algorithm and the other eight models were built from the parameters estimated using PSO. The experiment shows that the performances of HMM models, using parameters estimated from PSO and Baum-Welch approach, are comparable. We suggest that PSO or a combination of PSO and Baum-Welch algorithm could be alternative approaches for the HMM parameters estimation.

Tables are an easy way to represent information in a structural form. Table recognition is important for the extraction of such information from document images. Usually, modern OCR systems provide textual information coming from tables without recognizing actual table structure. However, recognition of table structure is important to get the contextual meaning of the contents. Table structure recognition in heterogeneous documents is challenging due to a variety of table layouts. It becomes harder where no physical rulings are present in a table. This work proposes a novel learning based methodology for the recognition of table contents in heterogeneous document images. Textual contents of documents are classified as table or non-table elements using a pre-trained neural network model. The output of the neural network is further enhanced by applying a contextual post processing on each element to correct the classifications errors if any. The system is trained using a subset of UNLV and UW3 document images and depicted more than 97% accuracy on a test set in detection of table and non-table elements.

We introduce YOLO9000, a state-of-the-art, real-time object detection system that can detect over 9000 object categories. First we propose various improvements to the YOLO detection method, both novel and drawn from prior work. The improved model, YOLOv2, is state-of-the-art on standard detection tasks like PASCAL VOC and COCO. Using a novel, multi-scale training method the same YOLOv2 model can run at varying sizes, offering an easy tradeoff between speed and accuracy. At 67 FPS, YOLOv2 gets 76.8 mAP on VOC 2007. At 40 FPS, YOLOv2 gets 78.6 mAP, outperforming state-of-the-art methods like Faster RCNN with ResNet and SSD while still running significantly faster. Finally we propose a method to jointly train on object detection and classification. Using this method we train YOLO9000 simultaneously on the COCO detection dataset and the ImageNet classification dataset. Our joint training allows YOLO9000 to predict detections for object classes that dont have labelled detection data. We validate our approach on the ImageNet detection task. YOLO9000 gets 19.7 mAP on the ImageNet detection validation set despite only having detection data for 44 of the 200 classes. On the 156 classes not in COCO, YOLO9000 gets 16.0 mAP. YOLO9000 predicts detections for more than 9000 different object categories, all in real-time.

Digital symbolic music scores offer many benefits compared to paper-based scores, such as a flexible dynamic layout that allows adjustments of size and style, intelligent navigation features, automatic page-turning, onthe-fly modifications of the score including transposition into a different key, and rule-based annotations that can save hours of manual work by automatically highlighting relevant aspects in the score. However, most musicians still rely on paper because they don’t have access to a digital version of their sheet music, or their digital solution does not provide a satisfying experience. To bring digital scores to millions of musicians, we at Enote are building a mobile application that offers a comprehensive digital library of sheet music. These scores are obtained by a large-scale Optical Music Recognition process, combined with metadata collection and curation. Our material is stored in the MEI format and we rely on Verovio as a central component of our app to present scores and parts dynamically on mobile devices. This combination of the expressiveness of MEI with the beautiful engraving of Verovio allows us to create a flexible, mobile solution that we believe to be a powerful and true alternative to paper scores with practical features like smart annotations or instant transpositions. We also invest heavily into the open-source development of Verovio to make it the gold standard for rendering beautiful digital sheet music.

There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .

The following sections are included: Introduction The Bentham Datasets Used in the HTR Contests Contest Protocol and Evaluation Methodology Existing State-of-the-Art Approaches HTR Approaches Employed in the Contests Presentation and Analysis of HTR Contest Results Remarks and Conclusions References

A new method is presented for adaptive document image binarization, where the page is considered as a collection of subcomponents such as text, background and picture. The problems caused by noise, illumination and many source type-related degradations are addressed. Two new algorithms are applied to determine a local threshold for each pixel. The performance evaluation of the algorithm utilizes test images with ground-truth, evaluation metrics for binarization of textual and synthetic images, and a weight-based ranking procedure for the final result presentation. The proposed algorithms were tested with images including different types of document components and degradations. The results were compared with a number of known techniques in the literature. The benchmarking results show that the method adapts and performs well in each case qualitatively and quantitatively.

This easy-to-follow guide helps you master the fundamentals of music notation and theory in no time.

We describe and validate a simple context-based scene recognition algorithm for mobile robotics applications. The system can differentiate outdoor scenes from various sites on a college campus using a multiscale set of early-visual features, which capture the "gist" of the scene into a low-dimensional signature vector. Distinct from previous approaches, the algorithm presents the advantage of being biologically plausible and of having low-computational complexity, sharing its low-level features with a model for visual attention that may operate concurrently on a robot. We compare classification accuracy using scenes filmed at three outdoor sites on campus (13,965 to 34,711 frames per site). Dividing each site into nine segments, we obtain segment classification rates between 84.21 percent and 88.62 percent. Combining scenes from all sites (75,073 frames in total) yields 86.45 percent correct classification, demonstrating the generalization and scalability of the approach

Not Available

In the last decades historical handwritten documents have become increasingly available in digital form. Yet, the accessibility to these documents with respect to browsing and searching remained limited as full automatic transcription is often not possible or not sufficiently accurate. This paper proposes a novel reliable approach for template-based keyword spotting in historical handwritten documents. In particular, our framework makes use of different graph representations for segmented word images and a sophisticated matching procedure. Moreover, we extend our method to a spotting ensemble. In an exhaustive experimental evaluation on four widely used benchmark datasets we show that the proposed approach is able to keep up or even outperform several state-of-the-art methods for template- and learning-based keyword spotting.

Segmentation of text from badly degraded document images is a very challenging task due to the high inter/intra-variation between the document background and the foreground text of different document images. In this paper, we propose a novel document image binarization technique that addresses these issues by using adaptive image contrast. The adaptive image contrast is a combination of the local image contrast and the local image gradient that is tolerant to text and background variation caused by different types of document degradations. In the proposed technique, an adaptive contrast map is first constructed for an input degraded document image. The contrast map is then binarized and combined with Canny's edge map to identify the text stroke edge pixels. The document text is further segmented by a local threshold that is estimated based on the intensities of detected text stroke edge pixels within a local window. The proposed method is simple, robust, and involves minimum parameter tuning. It has been tested on three public datasets that are used in the recent document image binarization contest (DIBCO) 2009 & 2011 and handwritten-DIBCO 2010 and achieves accuracies of 93.5%, 87.8%, and 92.03%, respectively, that are significantly higher than or close to that of the best-performing methods reported in the three contests. Experiments on the Bickley diary dataset that consists of several challenging bad quality document images also show the superior performance of our proposed method, compared with other techniques.

Libraries are positioned at the nexus of creative production, music publishing, performance, and research. The academic library community has the potential to play an influential leadership role in shaping the music publishing life cycle, making scores more readily discoverable and accessible, and establishing itself as a force that empowers a wide range of creativity and scholarship. Yet the music publishing industry has been slow to capitalize on the digital market, and academic libraries have been slow to integrate electronic music scores into their collections. In this paper, I will discuss the historical, technical, and human factors that have contributed to this moment, and the critical next steps the academic library community can take in response to the booming digital music publishing market to make a lasting impact through setting technological standards and best practices, developing education in these technologies and related intellectual property issues, and becoming an active partner in digital music publishing and in innovative research and creative possibilities.

This paper presents a methodology for evaluation of low-level image analysis methods, using binarization (two-level thresholding) as an example. Binarization of scanned gray scale images is the first step in most document image analysis systems. Selection of an appropriate binarization method for an input image domain is a difficult problem. Typically, a human expert evaluates the binarized images according to his/her visual criteria. However, to conduct an objective evaluation, one needs to investigate how well the subsequent image analysis steps will perform on the binarized image. We call this approach goal-directed evaluation, and it can be used to evaluate other low-level image processing methods as well. Our evaluation of binarization methods is in the context of digit recognition, so we define the performance of the character recognition module as the objective measure. Eleven different locally adaptive binarization methods were evaluated, and Niblack's method gave the best performance.

This paper presents an evaluation of locally adaptive binarization methods for gray scale images with low contrast, variable background intensity and noise. Such low quality occurs frequently in utility maps and excludes the use of global binarization methods. Only robust locally adaptive binarization methods with no need for on-line tuning of the parameters were considered since the gray scale images of utility maps often consist of a billion (10⁹) pixels or more. Eight locally adaptive binarization methods were tested on five different images. The postprocessing step (PS) of Yanowitz and Bruckstein's (1989) method improved all the other best binarization methods. Niblack's (1986) method with PS gave the best performance. Eikvil, Taxt and Moen's (1991) method with PS, and Yanowitz and Bruckstein's method did almost as well. Comparison was also made on the CPU requirement

This paper addresses the problem of generating possible object locations for use in object recognition. We introduce selective search which combines the strength of both an exhaustive search and segmentation. Like segmentation, we use the image structure to guide our sampling process. Like exhaustive search, we aim to capture all possible object locations. Instead of a single technique to generate possible object locations, we diversify our search and use a variety of complementary image partitionings to deal with as many image conditions as possible. Our selective search results in a small set of data-driven, class-independent, high quality locations, yielding 99 % recall and a Mean Average Best Overlap of 0.879 at 10,097 locations. The reduced number of locations compared to an exhaustive search enables the use of stronger machine learning techniques and stronger appearance models for object recognition. In this paper we show that our selective search enables the use of the powerful Bag-of-Words model for recognition. The selective search software is made publicly available (Software: http://disi.unitn.it/ uijlings/SelectiveSearch.html ).

Training the models needed for Automatic Handwritten Text Recognition of historical documents generally requires a significant amount of human effort. This is mainly due to the great differences that often exist between collections and to the lack of linguistic resources from the period when the documents were written, which results in a need of manual data labelling effort. This paper presents a study on the reuse of models trained with data from a different collection, focusing on the contribution that the language model and the optical models have on the performance. An empirical evaluation is performed using data from Jeremy Bentham manuscripts with the aim of recognising a manuscript about a very different topic written by Jane Austen.

This paper presents a new method for degraded-document binarization, inspired by the attributes of the Human Visual System (HVS). It can deal with various types of degradations, such as uneven illumination, shadows, low contrast, smears, and heavy noise densities. The proposed algorithm combines the characteristics of the OFF center-surround cells of the HVS with the classic Otsu binarization technique. Cells of two different scales are combined, increasing the efficiency of the algorithm and reducing the extracted noise in the final output. A new response function, which regulates the output of the cell according to the local contrast and the local lighting conditions is also introduced. The Otsu technique is used to binarize the outputs of the OFF center-surround cells. Quantitative experiments performed on a set of various computer-generated degradations, such as noise, shadow, and low contrast demonstrate the superior performance of the proposed method against six other well-established techniques. Qualitative and OCR comparisons also confirm these results.

The Web and other digital technologies have democratised music creation, reception, and analysis, putting music in the hands, ears, and minds of billions of users. Music digital libraries typically focus on an essential subset of this deluge—commercial and academic publications, and historical materials—but neglect to incorporate contributions by scholars, performers, and enthusiasts, such as annotations or performed interpretations of these artifacts, despite their potential utility for many types of users. In this paper we consider means by which digital libraries for musicology may incorporate such contributions into their collections, adhering to principles of FAIR data management and respecting contributor rights as outlined in the EU’s General Data Protection Regulation. We present an overview of centralised and decentralised approaches to this problem, and propose hybrid solutions in which contributions reside in a) user-controlled personal online datastores, b) decentralised file storage, and c) are published and aggregated into digital library collections. We outline the implementation of these ideas using Solid, a Web decentralisation project building on W3C standard technologies to facilitate publication and control over Linked Data. We demonstrate the feasibility of this approach by implementing prototypes supporting two types of contribution: Web Annotations describing or analysing musical elements in score encodings and music recordings; and, music performances and associated metadata supporting performance analyses across many renditions of a given piece. Finally, we situate these ideas within a wider conception of enriched, decentralised, and interconnected online music repositories.

Abstract Gradient descent training of neural networks can be done in either a batch or on-line manner. A widely held myth in the neural network community is that batch training is as fast or faster and/or more ‘correct’ than on-line training because it supposedly uses a better approximation of the true gradient for its weight updates. This paper explains why batch training is almost always slower than on-line training—often orders of magnitude slower—especially on large training sets. The main reason is due to the ability of on-line training to follow curves in the error surface throughout each epoch, which allows it to safely use a larger learning rate and thus converge with less iterations through the training data. Empirical results on a large (20,000-instance) speech recognition task and on 26 other learning tasks demonstrate that convergence can be reached significantly faster using on-line training than batch training, with no apparent difference in accuracy.

Abstract Machine recognition of a handwritten mathematical expression (HME) is challenging due to the ambiguities of handwritten symbols and the two-dimensional structure of mathematical expressions. Inspired by recent work in deep learning, we present Watch, Attend and Parse (WAP), a novel end-to-end approach based on neural network that learns to recognize {HMEs} in a two-dimensional layout and outputs them as one-dimensional character sequences in LaTeX format. Inherently unlike traditional methods, our proposed model avoids problems that stem from symbol segmentation, and it does not require a predefined expression grammar. Meanwhile, the problems of symbol recognition and structural analysis are handled, respectively, using a watcher and a parser. We employ a convolutional neural network encoder that takes {HME} images as input as the watcher and employ a recurrent neural network decoder equipped with an attention mechanism as the parser to generate LaTeX sequences. Moreover, the correspondence between the input expressions and the output LaTeX sequences is learned automatically by the attention mechanism. We validate the proposed approach on a benchmark published by the {CROHME} international competition. Using the official training dataset, {WAP} significantly outperformed the state-of-the-art method with an expression recognition accuracy of 46.55% on {CROHME} 2014 and 44.55% on {CROHME} 2016.

The use of object proposals is an effective recent approach for increasing the computational efficiency of object detection. We propose a novel method for generating object bounding box proposals using edges. Edges provide a sparse yet informative representation of an image. Our main observation is that the number of contours that are wholly contained in a bounding box is indicative of the likelihood of the box containing an object. We propose a simple box objectness score that measures the number of edges that exist in the box minus those that are members of contours that overlap the box's boundary. Using efficient data structures, millions of candidate boxes can be evaluated in a fraction of a second, returning a ranked set of a few thousand top-scoring proposals. Using standard metrics, we show results that are significantly more accurate than the current state-of-the-art while being faster to compute. In particular, given just 1000 proposals we achieve over 96 over 75 runs in 0.25 seconds and we additionally demonstrate a near real-time variant with only minor loss in accuracy.