User Interface (UI) testing is a popular approach to ensure the quality of mobile apps. Numerous test generation tools have been developed support UI on apps, especially for Android Previous work evaluates and compares different using only relatively simple open-source while real-world industrial apps tend more complex functionalities implementations. There no direct comparison among with regard effectiveness ease-of-use these To address such limitation, we study existing state-of-the-art or...

Given the ever increasing number of research tools to automatically generate inputs test Android applications (or simply apps), researchers recently asked question "Are we there yet?" (in terms practicality tools). By conducting an empirical study various tools, found that Monkey (the most widely used tool this category in industrial practices) outperformed all they studied. In paper, present two significant extensions study. First, conduct first case applying against WeChat, a popular...

Abstract Database systems play an important role in nearly every modern organization, yet relatively little research effort has focused on how to test them. This paper discusses issues arising testing database systems, presents approach applications, and describes AGENDA, a set of tools facilitate the use this approach. In such state before after user's operation plays role, along with input system output. A framework for applications is introduced. complete tool set, based framework, been...

Monkey, a random testing tool from Google, has been popularly used in industrial practices for automatic test input generation Android due to its applicability variety of application settings, e.g., ease use and compatibility with different platforms. Recently, Monkey under the spotlight research community: recent studies found out that none studied tools academia were actually better than when applied on set open source apps. Our efforts performed first case study applying WeChat, popular...

It is vital for popular mobile apps with large numbers of users to release updates rich features while keeping stable user experience. Timely and accurately locating emerging app issues can greatly help developers maintain update apps. User feedback (i.e., reviews) a crucial channel between users, delivering stream information about bugs that concern users. Methods identify based on have been proposed in the literature, however, their applicability industry has not explored. We apply recent...

AGENDA is a tool set for testing relational database applications. An earlier prototype was targeted to applications consisting of single query and included components populating with data suitable the application, generating inputs query, checking relatively simple aspects results executing query. This paper describes substantial extensions AGENDA, allowing it test transactions multiple queries complex intended behavior. The introduces technique properties state transition performed by...

Mobile applications, or apps for short, are gaining popularity. The input sources (e.g., touchscreen, sensors, transmitters) of the smart devices that host these enable to offer a rich experience users, but pose testing complications developers writing tests accurately utilize multiple together and be able replay such at later time). To alleviate complications, researchers practitioners in recent years have developed variety record-and-replay tools support expressiveness devices. These allow...

GUI testing is an essential part of regression for Android apps. For to remain effective, it important that obsolete test scripts get repaired after the app has evolved. In this paper, we propose a novel approach named GUIDER automated repair The key novelty lies in utilization both structural and visual information widgets on GUIs better understand what base version become updated version. A supporting tool been implemented approach. Experiments conducted popular messaging social media...

Code completion has become a common practice for programmers during their daily programming activities. It aims at automatically predicting the next tokens or lines that tend to use. A good code tool can substantially save keystrokes and improve efficiency programmers. Recently, various techniques have been proposed usage in practice. However, it is still unclear what are practitioners' expectations on whether existing research met demands. To fill gap, we perform an empirical study by first...

Navigating mobile User Interface (UI) applications using large language and vision models based on high-level goal instructions is emerging as an important research field with significant practical implications, such digital assistants automated UI testing. To evaluate the effectiveness of existing in navigation, benchmarks are required widely used literature. Although multiple have been recently established for evaluating functional correctness being judged pass or fail, they fail to...

Large-scale Android apps that provide complex functions are gradually becoming the mainstream in app markets. They tend to display many GUI widgets on a single page, which, unfortunately, can cause more redundant test actions—actions with similar functions—to automatic testing approaches. The effectiveness of existing approaches is still limited, suggesting necessity reducing effort actions. In this paper, we first identify three types structures actions and then propose novel approach,...

Modern systems generate a massive amount of logs to detect and diagnose system faults, which incurs expensive storage costs runtime overhead. After investigating real-world production logs, we observe that most the logging overhead is due small number log templates, referred as hotspots. Therefore, conduct systematical study about hotspots in an industrial WeChat, motivates us identify reduce them on fly. In this paper, propose LogReducer, non-intrusive language-independent reduction...

Commercial, scientific, and social activities are increasingly becoming dependent on Web database applications. New testing techniques that handle the unique features of these systems needed. To end, we have extended AGENDA, a tool set for relational applications, to test web Application source code is analyzed extract relevant information about URLs their parameters. This used construct simplify graph in which nodes represent edges links between URLs. A paths through selected cases...

Neural Machine Translation (NMT) has shown great advantages and is becoming increasingly popular. However, in practice, NMT often produces unexpected translation failures its translations. While reference-based black-box system testing been a common practice for quality assurance during development, an critical industrial named in-vivo testing, exposes unseen types or instances of when real users are using deployed system. To fill the gap lacking test oracles systems, we propose new...

Code completion has become a common practice for programmers during their daily programming activities. It automatically predicts the next tokens or statements that may use. aims to substantially save keystrokes and improve efficiency programmers. Although there exists substantial research on code completion, it is still unclear what practitioner expectations are whether these met by existing research. To address questions, we perform study first interviewing 15 professionals then surveying...

Built on the WeChat social platform, Mini-Programs are widely used by more than 400 million users every day. Consequently, reliability of is particularly crucial. However, suffer from various bugs related to execution environment, lifecycle management, asynchronous mechanism, etc. These have seriously affected users' experience and caused serious impacts.

Database application programs are often designed to be executed concurrently by many users. By grouping related database queries into transactions, DBMS (database management system) can guarantee that each transaction satisfies the well-known ACID properties: atomicity, consistency, isolation, and durability. However, if a is decomposed transactions in an incorrect manner, may fail when due potential offline concurrency problems. This paper presents dataflow analysis technique for...

Database systems play an important role in nearly every modern organization, yet relatively little research effort has focused on how to test them. AGENDA, A (test) GENerator for Applications, is a prototype tool set testing DB application programs. In such applications, the states of database before and after execution role, along with user's input system output. AGENDA components populate database, generate inputs, check aspects correctness output new state.

Lexical-based metrics such as BLEU, NIST, and WER have been widely used in machine translation (MT) evaluation. However, these badly represent semantic relationships impose strict identity matching, leading to moderate correlation with human judgments. In this paper, we propose a novel MT automatic evaluation metric Semantic Travel Distance (STD) based on word embeddings. STD incorporates both lexical features (word embeddings n-gram order) into one metric. It measures the distance between...

Code generation aims to synthesize code and fulfill functional requirements based on natural language (NL) specifications, which can greatly improve development efficiency. In the era of large models (LLMs), (LCMs) have been recently proposed generate source code. LCMs highly feasible solutions for programming problems described in language. Despite effectiveness, we observe a noticeable multilingual bias performance LCMs. Specifically, demonstrate proficiency generating when provided with...

Many analytical and empirical studies of software testing effectiveness have used the probability that a test set exposes at least one fault as measure effectiveness. That is useful for evaluating techniques when goal to gain confidence program free from faults. However, if improve reliability (by discovering removing those faults are most likely cause failures in field) then must distinguish between unlikely do so. Delivered was previously introduced means comparing setting. This paper...

Despite getting widely adopted recently, a Neural Machine Translation (NMT) system is often found to produce translation failures in the outputs. Developers have been relying on in-house testing for quality assurance of NMT. This methodology requires human-constructed reference translations as ground truth (test oracle) example natural language inputs. The has shown benefits quickly enhancing an NMT early development stages. However, industrial settings, it desirable detect without reliance...

Large-scale online systems are complex, fast-evolving, and hardly bug-free despite the testing efforts. Backend system monitoring cannot detect many types of issues, such as UI related bugs, bugs with small impact on backend indicators, or errors from third-party co-operating systems, etc. However, users good informers issues: They will provide their feedback for any issues. This experience paper discusses our design iFeedback, a tool to perform real-time issue detection based user texts....

JavaScript is one of the most popular programming languages. WeChat Mini-Program a large ecosystem applications that runs on platform. Millions Mini-Programs are accessed by users every week. Consequently, performance and robustness particularly important. Unfortunately, many suffer from various defects problems. Dynamic analysis useful technique to pinpoint application defects. However, due dynamic features language complexity runtime environment, techniques were rarely used improve quality...

WeChat, a widely-used messenger app boasting over 1 billion monthly active users, requires effective quality assurance for its complex features. Record-and-replay tools are crucial in achieving this goal. Despite the extensive development of these tools, impact waiting time between replay events has been largely overlooked. On one hand, long executing on fully-rendered GUIs slows down process. other short can lead to partially-rendered GUIs, negatively affecting effectiveness. An optimal...