Reading Seminar
2025
(Discussion) Recent studies have shown that Large Language Model (LLM)-based agents are vulnerable to security risks; that is, malicious prompts can exploit sensitive operations. To address this, the paper introduces AgentFuzz, a greybox fuzzing framework designed to detect taint-style vulnerabilities in LLM-based agents. AgentFuzz leverages dataflow analysis to identify predefined sinks, such as SQL and code injection, and constructs call chains that guide LLM-assisted prompt generation, forming the main seed pool. Afterward, each seed's execution trace is evaluated at both semantic and distance levels to schedule the seed to mutate first. Using this approach, AgentFuzz uncovered 34 previously unknown high-risk vulnerabilities, 23 of which were assigned CVE at the time of publication. This work is a novel direction for identifying vulnerabilities in LLM-based agents. (Jiyong).
(Discussion) This paper proposes a hybrid framework that combines Code Property Graphs (CPGs) with Large Language Models (LLMs) to improve software vulnerability detection. By constructing vulnerability-focused code slices, the system enables LLMs to capture security-critical context across both function-level and project-level codebases. Empirical evaluation demonstrates improvements over state-of-the-art baselines, strong robustness against code transformations, and effective generalization to unseen datasets. However, limitations remain, including the inability of CPG-based methods to capture runtime-dependent vulnerabilities, the scarcity of high-quality datasets, and context length constraints in the fine-tuned LLMs. In addition, the system’s performance degrades on deeply nested code structures and shows reduced effectiveness on underrepresented CWEs (Shakhzod).
(Discussion) This paper introduces EvoCrawl, a novel web application crawler that leverages evolutionary search to overcome the limitations of traditional state exploration in vulnerability detection. Its key strengths lie in the innovative use of dependency tracking and a fitness-based evolutionary algorithm, which enable it to efficiently navigate complex interaction sequences, achieve significantly higher code coverage, and uncover real zero-day vulnerabilities across widely used platforms. The implementation is technically robust, integrating multiple modules (ESM, PM, IVD, XVD) and demonstrating practical effectiveness through extensive case studies. Nonetheless, the work has some shortcomings: the design section is at times confusing in its presentation, terminology inconsistencies appear, and the system currently requires manual parameter tuning while struggling with certain token-handling scenarios. Despite these limitations, EvoCrawl represents a meaningful advancement in automated security testing by showing how evolutionary algorithms can push the boundaries of web application exploration and vulnerability discovery. (Suyeon)
(Discussion) This paper conducts the first empirical study of Rust-for-Linux, revealing both its successes and compromises. Its strengths lie in a thorough analysis of drivers, commits, and benchmarks, showing that Rust improves code quality, testing, and developer engagement while making Linux more securable. However, unsafe code remains inevitable, performance is inconsistent across drivers, and the steep learning curve complicates adoption. Despite these limitations, the study marks an important step in understanding how Rust reshapes kernel development, offering valuable insights for balancing safety, efficiency, and community growth in future RFL work. (Yiyue)
(Discussion) KernelGPT takes a stepwise approach to interpreting kernel driver and socket code with an LLM—recovering identifiers, types, and dependencies—and iteratively verifying and repairing its outputs to produce syzlang specifications, thereby extending fuzzing into paths that earlier tools left uncovered. In evaluation, it generated 532 new syscall specifications and 294 new type definitions; combined with Syzkaller, these additions unlocked 20,472 additional unique basic blocks and surpassed state-of-the-art baselines in both coverage and crash discovery. This specification lift translated into practice: the system uncovered 24 new kernel bugs (21 confirmed, 11 assigned CVEs, 12 already patched), underscoring a clear link between specification quality and fuzzing effectiveness. Still, the current pipeline relies largely on syntactic and shallow semantic checks and can miss runtime semantic errors, while risks such as LLM hallucination and version drift remain.
(Discussion) This paper analyzed and discovered vulnerabilities in email system attachment detection. It demonstrated that malicious code detection can be evaded by exploiting differences in MIME parsing methods between email security detectors and clients. Using this approach, they developed a framework and discovered a total of 24 vulnerabilities across 7 popular email clients. Although there are limitations such as potential errors in many real-world cases due to reliance on open-source parser filtering, and the approach is only applicable within restricted scenarios, they achieved significant results including being the first to analyze MIME parsing ambiguity and actually discovering vulnerabilities.(Hyoungjun)
(Discussion) This paper presents Coda, a neural-based framework for program decompilation that departs from traditional rule-based approaches by employing a two-phase design: code sketch generation using an instruction type-aware encoder and AST decoder, followed by iterative error correction guided by ensembled error predictors and compiler feedback. The framework achieves significantly higher program recovery accuracy than both conventional decompilers and Seq2Seq-based models, demonstrating its ability to preserve semantics as well as functionality. However, its effectiveness is constrained by assumptions such as access to compiler configurations and unoptimized binaries (-O0), and performance drops on more complex ISAs like x86-64 or longer programs due to input length and architectural challenges. While the study convincingly shows the feasibility of end-to-end neural decompilation and raises concerns about software IP protection, its applicability to real-world scenarios with optimized binaries, diverse architectures, and richer data structures remains an open challenge. (Mijin)
(Discussion) The study explores how OAuth 2.0, when applied in integration platforms like workflow automation, virtual assistants, and smart homes, can be exploited due to a role reversal in trust relationships. The authors identify two novel cross-app attacks, Cross-app OAuth Account Takeover (COAT) and Cross-app OAuth Request Forgery (CORF) that allow malicious apps to hijack or forge account linkages, sometimes with just a single click. A strength of this work is that it introduces a new threat model and attack types while keeping the scenarios realistic, and the authors also discovered real bugs in widely used platforms, even leading to bug bounty rewards. On the other hand, all attacks rely on the assumption that a victim connects to a malicious app, which is a relatively strong premise, and some of the proposed defenses are challenging to put into practice. Still, the contribution is meaningful in showing the prevalence of these vulnerabilities across major platforms such as Microsoft, Google, and Amazon, and in providing systematic analysis and concrete guidance for securing the ecosystem. (Suyeon)
(Discussion) This paper presents a forensic framework for investigating conversational AI services—specifically ChatGPT, Gemini, Copilot, and Claude—by treating them as forensic targets rather than tools. It analyzes how these services store and handle data across desktop apps, mobile apps, and web browsers, identifying various artifacts such as chat logs, cache files, attachments, and account information. The authors propose a five-phase forensic process (Preparation, Identification, Collection, Analysis, and Reporting) and provide open-source tools to support data extraction and normalization. Through detailed technical analysis and two case studies—a ransomware investigation and a copyright dispute—the paper demonstrates how forensic investigators can uncover evidence even from deleted or inaccessible accounts. However, the study has limitations: it relies heavily on platform-specific behaviors that may change over time and cannot recover deleted data without access credentials or device-level traces. Additionally, the specific forensic platform used is not clearly stated, and key assumptions for the investigation process are not explicitly defined. (Shakhzod)
(Discussion) This paper presents LLM4Decompile, an open-source series (1.3B–33B) tailored for binary-to-C decompilation via two complementary pipelines—an end-to-end model trained on disassembled ASM and a refinement model that cleans Ghidra output—together with pragmatic training choices (O0–O3 augmentation, duplicate filtering, and a two-stage “compilable→executable” curriculum). Its chief strengths are careful experimental design (per-optimization analysis, ablations disentangling compilable vs. executable distributions), clear, execution-based metrics, and a convincing demonstration that coupling LLMs with classical tooling (Ghidra+Ref) lifts re-executability beyond both GPT-4o and Ghidra alone; the obfuscation study is also a thoughtful, policy-aware addition. However, several weaknesses temper the claims: (i) external validity—the main targets are single functions or small C snippets on x86; results may not transfer to multi-file projects, other ISAs, or other languages; (ii) the key metric (re-executability under unit tests) can overestimate semantic recovery and underreport subtle miscompilations, while the GPT-based readability score introduces subjectivity; (iii) the “end-to-end” path still relies on disassembly (Objdump), sidestepping raw-byte modeling challenges; (iv) the refinement data decompiled from non-executable objects may diverge from real analyst workflows; and (v) scaling conclusions are confounded by the undertrained 33B model and limited compute, with no detailed error taxonomy or human-in-the-loop studies to gauge usefulness in practice. Overall, the work is a substantial step toward LLM-assisted decompilation and a strong empirical baseline, but broader architectures, languages, and user-centered evaluations will be essential to cement its impact. (Minseok)
(Discussion) This paper introduces JiT, an information-theoretic framework for zero-shot machine unlearning, addressing the challenge of removing specific training data from models without access to retained data. The authors propose a novel method that estimates a sample’s influence through local information gain, measured by the curvature of the model’s output space. By minimizing gradients in the neighborhood of forget samples, JiT effectively erases their impact while preserving model performance. The method outperforms prior zero-shot techniques across full-class, sub-class, and random unlearning benchmarks, achieving comparable performance to retraining with significantly lower computational cost. For instance, JiT maintains high accuracy on retained data and achieves similar entropy distributions to retrained models on CIFAR-10. However, the approach has notable limitations, including sensitivity to hyperparameter tuning, incompatibility with batch normalization, and lack of formal unlearning certification. JiT marks a significant advance in practical zero-shot unlearning, offering a scalable and effective method under strict constraints, yet highlights ongoing challenges in reliable data removal from black-box models.(Yiyue)
(Discussion) This paper proposes URL Inspection Tasks, an active defense mechanism to reduce phishing success in emails by requiring users to engage with a URL before visiting it. Inspired by CAPTCHAs, the system presents one of three tasks—clicking the correct domain, highlighting it, or re-typing it—designed to focus user attention on the domain and verify alignment with their intent. In a large-scale online study (2,673 participants), these tasks cut phishing success from 74.5% (control) to 35%, with particularly strong performance against typosquatting (down to 17.1%). However, the approach has limitations, including minor increases in false positives, added user burden (7–10 seconds), and reduced effectiveness for users unfamiliar with URL structures or non-Latin scripts. Despite these trade-offs, the work demonstrates that lightweight, interactive tasks can meaningfully reduce phishing risks by enhancing user awareness.(Jiyong)
(Discussion) This paper presents a security-focused analysis of Google's Agent-to-Agent (A2A) protocol, a foundational framework for enabling secure communication in agentic AI systems. As autonomous agents become more complex and interconnected, traditional security models fall short. The authors apply the MAESTRO threat modeling framework—tailored for multi-agent AI—to identify key vulnerabilities in A2A systems, including Agent Card spoofing, task replay attacks, and prompt injection via poisoned metadata. Through two detailed case studies, the paper illustrates real-world risks and proposes robust mitigation strategies such as strict schema validation, mutual TLS, digital signatures, and secure Agent Card discovery. While offering actionable guidance and best practices for secure implementation, the work's effectiveness relies heavily on disciplined protocol adherence and may face scalability challenges in untrusted ecosystems. Nonetheless, it marks a critical step in building secure, trustworthy infrastructure for the next generation of interoperable agentic applications. (Yujeong)
(Discussion) This paper introduces the Model Context Protocol (MCP), a standardized framework that streamlines interaction between AI models and external tools, addressing long-standing inefficiencies in tool integration, manual API wiring, and fragmented plugin systems. Inspired by the Language Server Protocol, MCP allows AI agents to dynamically discover and invoke external tools through a unified interface, enhancing autonomy and flexibility. The paper presents a comprehensive overview of the MCP ecosystem, detailing its architecture—comprising MCP hosts, clients, and servers—as well as the protocol's workflow and server lifecycle phases: creation, operation, and update. Notably, the authors conduct the first systematic analysis of MCP's security landscape, identifying risks such as name collision, installer spoofing, sandbox escapes, and privilege persistence. With real-world adoption by industry leaders like OpenAI, Cloudflare, and Anthropic, MCP is rapidly becoming foundational for AI-native applications. However, its ecosystem remains immature and decentralized, lacking formal security standards, version control, and authentication protocols. The paper emphasizes the urgent need for centralized package management, secure installation frameworks, and improved debugging tools to support MCP's safe and scalable growth. While MCP marks a significant step toward enabling agentic AI workflows, its current vulnerabilities underscore the importance of proactive governance, rigorous security audits, and continued research into robust deployment mechanisms. (Yujeong)
(Discussion) This paper presents TYPEPULSE, a static analysis tool for detecting type confusion bugs in Rust programs, addressing a critical gap in memory safety analysis for Rust's unsafe code. Unlike C/C++, Rust enforces safety through strict compile-time checks, but unsafe pointer conversions can still lead to severe bugs. TYPEPULSE targets three key bug types—misalignment, inconsistent layout, and mismatched scope—by analyzing type conversions, trait-bound generics, and pointer aliasing. It builds a property graph to capture interprocedural behaviors and uses this to identify invalid type accesses. Evaluated on 3,000 top Rust packages, TYPEPULSE uncovered 71 previously unknown bugs, surpassing the number reported to RustSec over the last five years. However, the tool's precision (75.5%) is limited by its reliance on static patterns and its difficulty in interpreting complex developer-enforced checks. Despite these limitations, TYPEPULSE represents a major step forward in automated detection of unsafe Rust bugs, offering both greater coverage and accuracy than existing tools like Clippy and Rudra. (Hyoungjun)
(Discussion) This paper analyzes Korea Security Applications (KSA) 2.0, mandatory security software for South Korean financial services, revealing that government-mandated security solutions can create more vulnerabilities than they prevent. The researchers identified four fundamental design flaws and 19 concrete vulnerabilities, including the paradoxical finding that anti-keylogging software could be exploited for keylogging attacks. With 97% of banking users having installed KSA despite 59% not understanding its functions, the study demonstrates how mandatory deployment amplifies security risks across national digital infrastructure. The vulnerabilities were actively exploited in North Korean cyberattacks in 2023, showing how mandatory software becomes attractive targets for nation-state actors. The remediation process revealed systemic issues where vendors resist fundamental security fixes due to backward compatibility concerns, creating perverse incentives that prioritize stability over security. The study serves as a cautionary tale for policymakers, emphasizing the need for rigorous security assessment before mandating any security technology. (Mijin)
(Discussion) This paper presents XDAC, a framework designed to detect and attribute LLM-generated comments in Korean news platforms, addressing the critical challenge of maintaining online discourse integrity in an era of sophisticated AI-generated content. The research tackles a significant gap in existing detection methods, which are inadequate for short-form content (Korean news comments average 51 characters versus GPTZero's 250-character minimum requirement). Using XAI-driven linguistic analysis, the authors identified distinguishing patterns between human and LLM content, such as LLMs' preference for formal structures versus humans' use of informal expressions and repeated characters. The framework achieves impressive performance with 98.5% F1 score in detection and 84.3% in attribution, successfully identifying 27,029 potential LLM-generated comments from 5.24M real-world comments on Naver. However, the approach faces significant limitations including reliance on artificially generated training data, heavy dependence on Korean-specific linguistic patterns that limit cross-lingual generalizability, and concerning vulnerabilities to adversarial attacks. The research represents a significant advancement in short-form content detection but highlights the ongoing arms race between generation and detection technologies, emphasizing the need for more robust defenses.(Mijin)
(Discussion) This paper proposes 'VulSim', a novel framework to solve the persistent problem of data scarcity in vulnerability detection. Instead of simply increasing data volume, VulSim enhances informational depth by integrating a code's semantic, contextual, and syntactic properties and analyzing the similarity of neighboring data. This unique approach surpassed existing state-of-the-art models with 75.41% accuracy on the Microsoft CodexGLUE benchmark and demonstrated excellent generalizability and practical potential by achieving a high recall of 85% on a completely new dataset. Despite these achievements, several clear limitations exist. First, as the model was validated only on C code, its effectiveness on the unique vulnerability patterns of other programming languages remains unknown. Second, its function-level analysis carries a methodological constraint, as it may miss complex vulnerabilities that span multiple functions or be diluted by irrelevant code. Finally, from a practical standpoint, the trade-off of sacrificing precision (53% on unseen data) for high recall can become a major obstacle to adoption, as it would incur significant costs in triaging false positives in a real-world environment. In conclusion, while VulSim presents an effective alternative beyond data-centric approaches, further research is needed to fully realize its potential. Future work must go beyond expansion to other languages and focus on resolving granularity issues and optimizing the precision-recall balance to enhance its reliability and efficiency in real-world development settings. (Intae)
(Discussion) This paper introduces AudioMarkNet, a proactive watermarking technique designed to embed imperceptible watermarks into real speech. Its goal is to prevent the misuse of genuine speech data for speaker adaptation in text-to-speech (TTS) systems—one of the main techniques used to generate realistic deepfake audio. Unlike traditional methods that detect deepfakes after they’re created, AudioMarkNet works preventively. The authors show that their method effectively detects fake speech from both open-source and commercial TTS models and remains robust even under non-adaptive and adaptive attacks. Code and examples are publicly available. Although the method is effective in detecting cloned voices, a limitation is that the watermarks may become noticeable during silent sections of the audio. This can allow adversaries to identify and potentially filter out watermarked speech (Shakhzod).
(Discussion) This paper introduces a novel technique called LLMmap for fingerprinting LLMs integrated into applications. Unlike previous methods that require access to internal model parameters or rely on watermarking, LLMmap uses an active querying strategy: it sends carefully crafted prompts to an application and analyzes the model's responses to identify its specific version. The authors demonstrate that LLMmap can accurately identify 42 popular LLM versions with over 95% accuracy using as few as 8 queries. The study also shows how fingerprinting enables more precise adversarial attacks and discusses the difficulty of building effective defenses, as LLM outputs are often distinct and hard to obfuscate consistently without degrading utility. However, the study has key limitations: it focuses solely on fingerprinting model outputs in interactive settings, while future work could extend to fingerprinting LLM-generated documents, images, or videos. Additionally, its approach to handling novel models is limited, new LLM versions are merely categorized as "unseen" without finer granularity to identify their exact variant.
(Discussion) The paper presents a large scale analysis of Telegram’s conspiracy theory ecosystem by combining a novel Conspiracy Resource Dataset drawn from prior academic work on YouTube, Reddit, and other fringe platforms with the TGDataset of 120,000 public Telegram channels. Using URL matching and community detection on a forward graph of 498 million messages, the authors identify four distinct "conspiracy communities" totaling nearly 18,000 channels, then reveal how these channels monetize their audiences through donations, crowdfunding, and affiliate links raising almost $71 million from over 900,000 backers. They also release two datasets and prototype mitigation tools (a Telegram bot and browser plug-in) to warn users about suspicious channels and links. However, the study’s reliance on link-based detection overlooks monetization embedded in media or private features (e.g., sponsored images, webinars, Telegram’s own Sponsored Messages), its Conspiracy Resource Dataset may be biased toward English language sources and known platforms (potentially under-detecting non-English communities), and the 9 to 15 month gap between message collection and monetization crawling could understate ongoing campaigns. Furthermore, using GPT-4o to classify promotional versus discrediting messages, while efficient, introduces misclassification risk, and the analysis stops short of firmly linking channel operators to the funds they promote issues that call for more comprehensive signal integration, finer grained validation, and evaluation of real-world efficacy.
(Discussion) Identifying vulnerabilities early is crucial for preventing security incidents. Moreover, the common weakness enumerations (CWEs) provide a useful taxonomy for organizing various vulnerabilities in code. Building on CWEs, this paper propose a hierarchical contrastive learning framework to classify code vulnerability types by clustering related weaknesses in a shared embedding space. While the approach shows promising results and outperforms existing baselines on their selected dataset, the evaluation raises few concerns. The model is only tested on a curated vulnerability dataset (ie., BigVul and PrimeVule), which, despite including non-vulnerable functions, may not reflect the complexity of large scale projects like the Linux kernel. Moreover, the model performs worse on PrimeVul -- a dataset with higher label accuracy -- suggesting limitations in a more realitic scenario. Although, the framwork demonstrates higher performance against state-of-the-art models, its application remains uncertain without further investigation.
(Discussion) TokenFormer introduces a novel approach by integrating a p-attention layer into a Transformer-based architecture. Unlike the standard self-attention mechanism, the p-attention layer stores its weights as external parameters. This design allows for more efficient fine-tuning on new datasets with reduced computational overhead. A key distinction of this method is that both the original attention weights "old parameters" and the newly introduced weights "new parameters" remain trainable during fine-tuning. This contrasts with many existing fine-tuning strategies, which typically freeze the original parameters to prevent catastrophic forgetting. Interestingly, the authors do not explicitly address catastrophic forgetting in the paper, despite deviating from the conventional practice of parameter freezing.
(Discussion) This paper tackles the challenge of factual unlearning in LLMs by proposing AltPO, a method that integrates negative feedback with in-distribution positive feedback to improve coherence and avoid nonsensical outputs. The approach is empirically validated on the TOFU benchmark and demonstrates strong performance across new and existing unlearning metrics. However, the paper assumes that alternate plausible answers do not risk introducing misinformation, which is not sufficiently addressed in the evaluation. Additionally, while AltPO is shown to generalize across different forget percentages, the method is limited to QA-style factual unlearning and may not transfer well to other data modalities or task types such as dialogue or reasoning.
(Discussion) This paper aims to infer function names from stripped binaries. To better identify function names, the authors applied a technique called Domain Adaptation for fine-tuning. Their proposed model, SymGen, demonstrated good performance in various experiments, particularly showing effectiveness with unseen data. However, the paper failed to indicate that the experimental datasets used cannot be considered truly unseen, and it did not acknowledge that what is referred to as "data leakage" was actually deliberately designed.
(Discussion) DeGPT presents an innovative approach to optimizing decompiler output, demonstrating significant improvements in code readability and supporting reverse engineering analysis through a unique three-role mechanism powered by Large Language Models (LLM). However, since LLM responses are not always accurate, the implementation of the MSSC technique is crucial for preventing changes in function semantics and ensuring the correctness of optimizations. Experimental results show that DeGPT performs exceptionally well across various optimization tasks, effectively enhancing variable renaming and comment addition, thus aiding in the understanding of binary code. Nonetheless, the metrics used in the experiments may not fully capture the subjective elements of code optimization, and there may be limitations in fully explaining its real-world impact on reverse engineering analysis. Despite these limitations, the research contributes a valuable framework that significantly enhances the reverse engineering process by improving the comprehensibility and effectiveness of decompiler output.
(Discussion) The paper introduces LeakLess, a novel approach to protect sensitive data within serverless platforms from memory disclosure and transient execution attacks. LeakLess employs in-memory encryption and a separate I/O module to manage cryptographic operations, ensuring sensitive data remains encrypted within the serverless runtime's memory. This I/O module acts as a secure intermediary, handling the decryption and encryption of sensitive data during communication between serverless functions and external entities. Implemented on the Spin serverless platform, LeakLess demonstrates robust protection with minimal performance overhead. However, the clarity of the result demonstration could be improved, as there might be a possibility of plain text partially remaining in the memory region. Additionally, the reliance on tools like gcore for memory analysis might not guarantee a comprehensive search of unmapped memory regions. While LeakLess offers a significant step towards enhancing the security of serverless applications, these aspects regarding the demonstration of results and the necessity for broader evaluation across different platforms warrant further investigation. The open-source nature of LeakLess encourages community involvement and wider adoption.
(Discussion) This paper presents a novel and practical membership inference attack (MIA) against Retrieval-Augmented Generation (RAG) systems, allowing adversaries to determine if specific documents exist in the retrieval database using only model outputs. The proposed black-box and gray-box attacks are tested on medical and email datasets across multiple generative models (Flan, LLaMA, Mistral), showing high effectiveness, especially in the gray-box setting. An initial defense, involving modified RAG templates with instruction-based filtering, significantly reduces attack success for LLaMA and Mistral but is largely ineffective against Flan. While the attack is simple and requires no internal access, its reliance on prompt crafting and limited defense evaluation suggest a need for more robust, adaptive privacy-preserving techniques in future work.
(Discussion) This paper introduces S2MIA, a novel membership inference attack that targets the external databases of Retrieval-Augmented Generation (RAG) systems by exploiting semantic similarity and perplexity between a target sample and the generated output. Unlike prior attacks relying on direct prompt responses or overfitting, S2MIA infers membership by measuring how closely the system's response matches the input, even under sampling variability. Experiments across five LLMs and two datasets show that S2MIA consistently outperforms five prior MIA methods, even when standard defenses like paraphrasing, prompt modification, and re-ranking are applied. The approach is notable for its simplicity, strong performance, and model-agnostic design. However, its effectiveness weakens under paraphrasing and prompt defenses, and it relies on BLEU scores and access to model outputs and perplexity, which may not be available in all RAG systems.
(Discussion) This paper introduces Mask-Based Membership Inference Attacks (MBA), a novel approach to determine whether a target document is stored in a Retrieval-Augmented Generation (RAG) system's knowledge base. The method involves masking challenging terms in the target document, using the masked version as a query, and then assessing whether the RAG system can correctly predict the masked terms. If the masked terms are accurately predicted, it suggests the original document is likely stored in the database. The approach outperforms prior MIA methods across multiple benchmarks, achieving significantly higher ROC AUC scores. Its strengths include robustness in black-box settings, strong empirical results, and resistance to noise from internal LLM knowledge. However, it relies on high retrieval quality, handcrafted masking strategies, and tuning of multiple hyperparameters (e.g., number of masks, prediction threshold), which may limit scalability or generalizability in diverse RAG settings.
(Discussion) The Chromium browser provides browser extensions to boost user experience. As a result, browsers like Chrome distribute browser extensions via their Web Store. However, the review system in the Chrome Web Store is subject to various malicious manipulations. This paper presents FakeX, a framework to detect fake reviews by analyzing the review metadata. FakeX employs five distinct methods (e.g., temporal distribution analysis, relationship clustering, and ratio-based assessment) to identify patterns indicative of fake reviews. The authors evaluate over 1.7 million reviews and identify hundreds of suspected fake review campaigns. Furthermore, the authors uncover 86 malicious extensions ranging from data stealing to monetization. The authors highlight the potential connection between some identified fake review campaigns and malicious extensions. However, the absence of a ground truth makes the validity of the method difficult to evaluate. Furthermore, the authors do not compare their work with any existing previous work which also handles fake reviews. Nevertheless, the identification of actual malicious extensions on top of their correlation with identified fake review campaigns is an interesting finding.
(Discussion) This paper introduces VERIBIN, a system for verifying whether binary-level patches maintain functional correctness. Since security patches are often distributed as binaries without source code, traditional source-code-based verification tools are insufficient. VERIBIN employs symbolic execution to compare original and patched binaries, ensuring that modifications do not break functionality. The system improves efficiency through the Matching Path Pairs (MPP) approach and enhances accuracy with adaptive verification, allowing analyst input when necessary. The evaluation on 86 real-world patches demonstrates 93% accuracy with 0% false positives, proving its effectiveness. The paper's key strength lies in its practical applicability and its emphasis on the critical need for binary patch verification, especially in light of incidents like the XZ Utils backdoor. However, symbolic execution can be resource-intensive, leading to timeouts and memory limitations on complex binaries. Additionally, while VERIBIN can handle many structural differences, functionally equivalent but syntactically different changes remain a challenge for full automation. Despite these limitations, VERIBIN represents a significant step forward in binary patch verification and has great potential to become a powerful automated security analysis tool with further improvements.
(Discussion) This paper investigates privacy risks in Cross-App Content Sharing (Cracs), revealing how sharing activities expose user identities, social relationships, and personal data. The authors identify three key privacy issues, Sharing Behavior Tracking (SBT), where source apps infer social connections; Sharing Data Interception (SDI), where shared content is secretly collected. And Sharer Data Exposure (SDE), where a sharer’s identity is unintentionally revealed to the receiver. Using Shark, an analysis tool combining static and dynamic methods, they find that over 55% of Chinese apps and 10% of US apps exhibit privacy vulnerabilities. While the paper highlights critical concerns, it has some weaknesses. The focus on Android apps limits generalizability to iOS, and Shark’s accuracy against obfuscated code is unclear. And the discussion on legal frameworks is superficial, and there is no exploration of real-world harm from these privacy leaks. Strengthening these areas would enhance the study’s impact.
(Discussion) This paper highlights critical blind spots in SIEM rule based detection, showing that 38% of Sigma rules can be fully evaded with simple obfuscation techniques. To address this, the authors propose AMIDES, a machine-learning-based detection system that identifies evasions by comparing events to existing SIEM rules. Tested on 155 million enterprise security events, AMIDES detects 70% of evasions with zero false positives and operates 330 times faster than required for real-time deployment. A major strength of this study is its real-world validation using large-scale enterprise data and its open-source implementation, making it highly practical and reproducible. It also expands detection beyond process creation logs to include PowerShell, registry, and web requests, increasing its applicability. However, AMIDES relies on supervised learning, making it vulnerable to novel, unseen evasions, and its long-term efficiency in evolving attack landscapes needs further testing. Despite these challenges, this work demonstrates the inadequacy of rule-based SIEM detection alone and underscores the need for hybrid approaches integrating AI-driven analytics. By identifying a major cybersecurity weakness and providing a scalable solution, the study contributes significantly to the future of SIEM systems and adaptive threat detection.
(Discussion) ERASAN introduces a selective instrumentation approach to address sanitization in Rust, significantly reducing ASan’s runtime overhead without compromising its ability to detect memory bugs. By targeting raw pointer-related vulnerabilities, ERASAN eliminates redundant checks and achieves substantial performance gains. The evaluation demonstrates impressive efficiency improvements, making it a promising alternative for Rust develo However, several concerns remain. The static analysis approach may overlook certain dynamic memory issues,its optimization strategy struggles with multi-threaded environments, as memory allocation patterns in concurrent execution cannot always be statically determined. Another challenge lies in the scope of evaluation, which focuses on popular Rust crates, leaving questions about its applicability in domains with heavy foreign function interface (FFI) usage or unconventional memory management patterns. Finally, the increased compile-time overhead may hinder adoption, particularly in large-scale projects requiring frequent recompilation. While ERASAN makes notable strides in improving Rust’s memory safety tools, its scalability, multi-threading limitations, and compilation costs warrant further refinement to maximize its practicality across diverse real-world applications.
(Discussion) This paper presents PHYFU, an innovative fuzzing framework aimed at detecting logic errors in physics simulation engines (PSES) by leveraging forward and backward simulation oracles grounded in physical laws. The authors demonstrate its effectiveness across eight configurations, uncovering various errors in both forward and backward simulation processes. However, the study has several limitations. First, the reliance on manual hyperparameter tuning for oracle thresholds and perturbation magnitudes makes the approach labor-intensive and less reproducible. Second, while PHYFU claims general applicability, its evaluation is confined to a limited set of scenarios and engines, leaving its effectiveness in diverse or unconventional physical simulations uncertain. Third, the framework's detection of "unapparent errors," which lack observable patterns, raises concerns about false positives and the robustness of its oracles. Finally, although PHYFU highlights errors caused by simulation inaccuracies, it does not explore their downstream impact on applications such as robotics or training agents, missing an opportunity to contextualize the significance of the findings. Despite its novelty, these constraints highlight challenges in scalability and broader applicability.
(Discussion) The study explores instruction-based backdoor attacks on large language models (LLMs), targeting their instruction-following capabilities through crafted prompts embedding backdoor instructions. It categorizes attacks into word-level, syntax-level, and semantic-level, showing high attack success rates (ASR) while maintaining clean input utility. Larger models like GPT-4 and Claude-3 exhibit higher ASRs, making them more susceptible than smaller models such as LLaMA2 and Mistral. However, the attack's effectiveness depends heavily on trigger design, including length, position, and complexity, with semantic-level attacks being stealthier but less versatile in simpler tasks. The study highlights challenges in defending against such attacks, as existing detection methods like intent analysis yield high false alarm rates and limited accuracy in identifying more covert triggers. Practical limitations include the reliance on static prompt structures, assuming attackers have precise knowledge of prompt formats and tasks, which may not generalize to dynamic or adversarial real-world scenarios. Additionally, the approach has not been tested extensively across diverse LLM architectures or specialized domains, leaving gaps in understanding broader vulnerabilities and mitigation strategies.
(Discussion) The paper introduces PLeak, an innovative framework designed to extract confidential system prompts from LLM applications, which are often protected as intellectual property. PLeak automates the process using a closed-box optimization approach, relying on shadow LLMs and datasets to simulate the target environment. It uses techniques like incremental query optimization and response aggregation to reconstruct system prompts effectively. The framework significantly outperforms previous methods, achieving high accuracy in metrics such as Exact Match and Semantic Similarity, and successfully reconstructs prompts in 68% of real-world tests on the Poe platform. However, PLeak has limitations, including its dependence on shadow environments that closely resemble the target, potential scalability issues with computationally intensive optimizations, and only partial exploration of defenses like output filtering or obfuscation. While it demonstrates critical security risks in LLM applications, the lack of broad evaluation across diverse platforms and limited engagement from reported parties like Poe raise concerns about its practical implications and ethical considerations.
2023
(Abstract) Binary analyses based on deep neural networks (DNNs), or neural binary analyses (NBAs), have become a hotly researched topic in recent years. DNNs have been wildly successful at pushing the performance and accuracy envelopes in the natural language and image processing domains. Thus, DNNs are highly promising for solving binary analysis problems that are hard due to a lack of complete information resulting from the lossy compilation process. Despite this promise, it is unclear that the prevailing strategy of repurposing embeddings and model architectures originally developed for other problem domains is sound given the adversarial contexts under which binary analysis often operates. In this paper, we empirically demonstrate that the current state of the art in neural function boundary detection is vulnerable to both inadvertent and deliberate adversarial attacks. We proceed from the insight that current generation NBAs are built upon embeddings and model architectures intended to solve syntactic problems. We devise a simple, reproducible, and scalable black-box methodology for exploring the space of inadvertent attacks – instruction sequences that could be emitted by common compiler toolchains and configurations – that exploits this syntactic design focus. We then show that these inadvertent misclassifications can be exploited by an attacker, serving as the basis for a highly effective black-box adversarial example generation process. We evaluate this methodology against two state-of-the-art neural function boundary detectors: XDA and DeepDi. We conclude with an analysis of the evaluation data and recommendations for ho
(Abstract) Taint analysis has been widely used in many security applications such as exploit detection, information flow tracking, malware analysis, and protocol reverse engineering. State-of-theart taint analysis tools are usually built atop dynamic binary instrumentation, which instruments at every possible instruction, and rely on runtime information to decide whether a particular instruction involves taint or not, thereby usually having high performance overhead. This paper presents SELECTIVETAINT, an efficient selective taint analysis framework for binary executables. The key idea is to selectively instrument the instructions involving taint analysis using static binary rewriting instead of dynamic binary instrumentation. At a high level, SELECTIVETAINT statically scans taint sources of interest in the binary code, leverages value set analysis to conservatively determine whether an instruction operand needs to be tainted or not, and then selectively taints the instructions of interest. We have implemented SELECTIVETAINT and evaluated it with a set of binary programs including 16 coreutils (focusing on file I/O) and five network daemon programs (focusing on network I/O) such as nginx web server. Our evaluation results show that the binaries statically instrumented by SELECTIVETAINT has superior performance compared to the state-of-the-art dynamic taint analysis frameworks (e.g., 1.7x faster than that of libdft).
2024
(Discussion) This paper introduces D-CAPTCHA, an innovative active defense mechanism against real-time deepfakes. Unlike traditional passive detection methods that rely on analyzing patterns or artifacts, D-CAPTCHA challenges the capabilities of deepfake models by requiring the attacker to perform tasks that are easy for humans but difficult for AI to replicate, such as humming a tune or speaking with emotion. Experimental results demonstrate that D-CAPTCHA significantly enhances detection accuracy, achieving 91-100% compared to state-of-the-art methods. While it offers long-term effectiveness and extensibility in combating audio-based deepfake threats, its practical application is limited to real-time scenarios, and it may impact user experience if not carefully implemented. Furthermore, it is worth noting a minor inconsistency in Table 1, where the acronym "R" is used for both Repeat Accent and Raspberry, which could be corrected for clarity. Overall, D-CAPTCHA represents a groundbreaking approach to addressing the growing threat of deepfake technology and highlights the potential for proactive AI-driven security solutions that adapt to evolving adversarial capabilities.
(Discussion) Since the advent of Large Language Models (LLMs), there has been increasing interest in the concept of 'Jailbreak Prompts' for these LLMs. Since LLMs are trained on a large corpus of data, they can generate harmful content. As a result, LLM services provide 'Aligned Language Models' that aim to prevent generating harmful content. The 'Jailbreak Prompts' are prompts that allow an adversary to bypass the defense and cause LLMs to generate harmful content. This paper conducts a measurement study on existing Jailbreak Prompt methods to answer three research questions: 'What are jailbreak strategies and how do they work?', 'How do humans develop and execute jailbreak attacks?', and 'How to automate the process of jailbreaking?'. While the paper overviews the current landscape of jailbreak prompts, the metrics defined in the paper (Expected Maximum Harmfulness (EMH) and Jailbreak Success Rate(JSR)) are difficult to comprehend. For instance, in Table 3 the authors present the metrics for pairs of jailbreak prompts and malicious queries. Unfortunately, it is difficult to comprehend the result as EMH does not seem to be normalized (exceeding 1.0 in certain cases) and the overall JSR is very low, causing the reader to question the effectiveness of the attack.
(Discussion) This paper delve into the innovative contributions of RustSan, assessing its implications in the Rust programming ecosystem. While the integration of AddressSanitizer into Rust programming through selective instrumentation significantly reduces runtime overhead, it also raises questions about the balance between optimization and comprehensiveness of memory safety checks. One needs to consider the scenarios where safe and overlapping memory regions are crucial yet may be overridden inadvertently in corner cases. And all hopes on static analysis for detection of unsafe blocks may fall short of considering dynamic interactions which only appear at runtime and thus present challenges in practical applications. While this may sound promising, performance gains shown in benchmarks will have to be investigated further in larger-scale applications with less predictable memory access patterns. Finally, though the approach taken by RustSan mitigates some of the inherent trade-offs between flexibility and safety in Rust, it remains important to establish the robustness of the tool in the face of complex programming paradigms and its ease of integration into existing workflows for developers. This prompts an ongoing evaluation of its long-term reliability and adaptability.
(Discussion) This paper makes a valuable contribution by systematically revealing undocumented APIs and their potential security exploits in popular “super apps” such as WeChat and TikTok. By leveraging both static and dynamic analysis, the authors’ tool, APIScope, successfully identifies hidden APIs and demonstrates their risks, thereby advancing our understanding of super app ecosystems and offering actionable guidance for platform vendors and developers. However, several limitations remain. The study focuses exclusively on Android versions of super apps using the V8 engine, narrowing the generalizability of its findings and excluding large market segments, like iOS platforms or super apps employing different JavaScript engines. Moreover, while APIScope identifies hidden APIs and validates certain vulnerabilities through test miniapps, it does not fully explore how organizational factors—such as development practices, code review processes, or internal security policies—contribute to these undocumented APIs. The paper also falls short of presenting robust, long-term defenses or systematic industry-level guidelines for mitigating undocumented APIs. A deeper integration of automated patching, secure coding frameworks, and clearer vendor-side standardization efforts would strengthen the practical impact. Future work should strive to broaden platform scope, thoroughly investigate root causes behind the emergence of undocumented APIs, and propose more comprehensive preventative strategies.
(Discussion) The paper introduces neural phishing, a method to extract sensitive information from large language models (LLMs) by injecting benign-looking poisoned data into their training datasets. The attack has three phases: poisoning during pretraining, memorization during fine-tuning, and information extraction during inference. Success rates range from 10% to 80%, depending on the model size, data duplication, and attacker knowledge. Larger and overtrained models are more vulnerable. Standard defenses like deduplication and differential privacy are ineffective. The work highlights significant privacy risks in LLMs and emphasizes the need for advanced defenses. While the study is significant, its limitations include reliance on poisoned data appearing before secrets in training, difficulty in fully preventing memorization of sensitive patterns, and the low likelihood of individuals leaking sensitive data directly to LLMs.
(Discussion) The paper presents a novel method to infer whether a document was included in the training dataset of large language models (LLMs). Using a black-box approach based on predicted token-level probabilities and normalization techniques, the authors develop a meta-classifier to distinguish between member and non-member documents. Evaluated on OpenLLaMA with datasets from Project Gutenberg and ArXiv, the method achieves AUC scores of 0.856 for books and 0.678 for papers, demonstrating its effectiveness. The study highlights that even small LLMs retain significant memorization, raising concerns about privacy and transparency in LLM training data. However, the approach relies on known datasets, limiting its applicability to proprietary models without accessible training data. It also assumes that non-member documents are similar but temporally distinct, which may not always hold. Additionally, the method's focus on black-box inference may not address vulnerabilities in fine-tuned or modified LLMs, warranting further research into privacy mitigation strategies.
(Discussion) This study effectively highlights a critical security vulnerability in containerized applications, revealing that 8.5% of analyzed Docker images contain sensitive information such as cryptographic keys or API secrets. The large-scale dataset and rigorous filtering enhance the reliability of findings, providing strong evidence that secret leakage is a systemic issue rather than isolated errors. This insight underscores the need for structural changes in the Docker paradigm, offering valuable recommendations such as improved secret scanning tools and stricter default configurations. However, the reliance on regex-based detection may limit the identification of unconventional leaks, and the inability to validate API secrets due to ethical constraints reduces the completeness of the analysis. Furthermore, the discussion could have delved deeper into human and organizational factors, such as user training, to address the root causes of these leaks. Despite these limitations, the study makes significant contributions by exposing the widespread impact of secret leakage, affecting over 275,000 Internet-facing hosts, and providing a foundation for collaborative efforts to mitigate these vulnerabilities. This research holds substantial value for improving container security practices and guiding future innovations in secret management.
(Discussion) After the advent of deep learning (DL) models, several practical attacks against DL have surfaced. Consequently, the research community developed a complementary vetting technique to detect such attacks. However, these techniques do not address how to retrieve these DL models for inspection. Existing state-of-the-art memory forensic techniques rely on static data structure recovery, which cannot recover complex data structures used in DL software stacks. Furthermore, ML frameworks employ advanced garbage collection and memory management optimization, which obfuscates in-memory objects. Moreover, the layer weights are stored in a consecutive GPU memory while the context of these objects is stored in the CPU data structure. Lastly, white-box analysis techniques must rehost the DL model in a live environment for inspection. This paper proposes AiP, an automated system for recovering DL models from meory and rehosting them into a live process. AiP requires no prior knoweldge of the particular model. In the evaluation using LISA, CIFAR-10, and IMDB dataset, AiP successfully recovered 30 different models and achieved 100% recover accuracy. The paper demonstrates its practicality by recovering popular Python-based frameworks (Pytorch and Tensorflow), however acknowledges the existance of other ML frameworks (e.g., HuggingFace).
(Discussion) This paper presents BUDAlloc, a novel one-time allocator (OTA) designed to address use-after-free (UAF) vulnerabilities by decoupling virtual address management from the kernel. Unlike conventional approaches, BUDAlloc shifts virtual memory operations to the user level, reducing system call overhead and improving performance, especially in multi-threaded applications. Using eBPF-based page fault handling, BUDAlloc efficiently manages virtual aliases and batches memory operations to minimize fragmentation. It offers two modes—detection and prevention—allowing users to prioritize either bug detection or performance. Evaluations show that BUDAlloc improves performance by 15% over DangZero and reduces memory overhead by 61% compared to FFmalloc, making it a scalable and practical solution for UAF bug detection without modifying existing binaries.
(Discussion) The paper, Exploring ChatGPT’s Capabilities on Vulnerability Management, examines ChatGPT's ability to assist in six stages of vulnerability management, utilizing a dataset of 70,346 samples. The authors compare ChatGPT's performance with state-of-the-art (SOTA) methods, assessing tasks like bug report summarization and vulnerability repair. While ChatGPT demonstrates promising potential in some areas, such as summarizing bug reports, challenges persist, particularly in patch correctness assessment. The paper's strengths include a comprehensive dataset and innovative exploration of prompt engineering techniques. However, it faces limitations in generalizing ChatGPT’s performance across all tasks, pointing to future research needs for refining its application in security tasks. This study is significant in highlighting ChatGPT’s growing role in automated vulnerability management, with the potential for advancing research in the domain.
(Discussion) This paper presents WEBRR (Web-based Enterprise Record and Replay), a novel forensic system designed for replaying and investigating web-based attacks in modern web environments. The authors address the limitations of existing forensic analysis tools in dealing with the complexity of modern web applications and browsers. WEBRR achieves deterministic replay of web attacks by leveraging the single-threaded nature of JavaScript and implementing a comprehensive recording and replay mechanism for various web components, including the Document Object Model (DOM), network requests, and asynchronous operations. The system is implemented as an extension to the Chromium browser and demonstrates high accuracy in replaying both benign websites and malicious attacks across multiple platforms (Linux, Windows, and Android). The authors evaluate WEBRR against existing solutions like WebCapsule, Mugshot, and RR, showing superior performance in replaying complex web attacks. The system exhibits low runtime overhead (median 2.94% increase in page load time) and reasonable storage requirements (17 MB per minute of browsing). While WEBRR has some limitations, such as incomplete support for certain web technologies (e.g., WebRTC, IndexedDB), the authors argue that these can be addressed with additional engineering effort. Overall, WEBRR represents a significant advancement in web forensics, enabling more accurate and interactive investigations of web-based attacks.
(Discussion) This paper presents a novel deepfake attack called Foice. This attack generates a synthetic voice of a victim using just a single face image, without requiring any voice samples. The generated voice is realistic enough to fool commercial voice authentication systems, such as those used by WeChat, Microsoft Azure, and other platforms. The core idea behind Foice is to learn the partial correlation between face and voice features, and then combine these with random face-independent voice features generated from a Gaussian distribution. The attack's effectiveness was demonstrated through real-world experiments on several authentication systems and voice assistants, where it showed high success rates, indicating significant vulnerabilities in these systems. The study emphasizes the importance of new defenses against such attacks, as current systems lack sufficient protection against deepfake threats based solely on facial images.
(Discussion) The paper provides a comprehensive analysis of the MITRE ATT&CK evaluations, aiming to address the limitations of the raw evaluation results. The paper's strengths lie in its introduction of new analysis methods, including whole-graph analysis and holistic assessments, to gain deeper insights into EDR system capabilities. The authors' reconstruction of attack scenarios and subsequent analysis shed light on EDR systems' attack reconstruction, behavior correlation, and overall detection performance. However, the paper could be improved by addressing the lack of access to crucial information like false positive alarm volume, response time, and raw data, which limits the scope of the analysis. Despite this limitation, the paper offers valuable contributions to the field by providing a systematic interpretation of MITRE ATT&CK evaluation results, aiding researchers, practitioners, and vendors in understanding and improving EDR systems.
(Discussion) Code coverage faces a major challenge in that it only reflects a small part of a program's structure, leaving some crucial program constructs uncovered. To address this issue, this work proposes data coverage for guided fuzzing - a technique that focuses on detecting novel constant data references and maximizing their coverage. Additionally, real-world fuzzing practices were optimized by classifying data access according to semantics and designing customized collection strategies. This is crucial because improper handling of constant data can significantly impact fuzzing throughput. To further improve fuzzing efficiency, novel storage and utilization techniques were developed. Finally, libFuzzer was enhanced with data coverage capabilities and submitted to Google's FuzzBench for evaluation. In this evaluation, the proposed approach outperformed many state-of-the-art fuzzers and achieved the best coverage score in the experiment. As a result, using this enhanced code coverage, 28 previously unknown bugs in OSS-Fuzz projects were discovered.
(Discussion) This paper presents RACING, a novel approach for efficient root cause analysis (RCA) in fuzzing. By leveraging counterexamples and reinforcement learning, RACING intelligently guides the fuzzing process to quickly identify the root cause of crashes. The experimental results demonstrate that RACING significantly outperforms the state-of-the-art technique, Aurora, in terms of both speed and accuracy. This work holds significant implications for improving the efficiency and effectiveness of vulnerability detection and analysis in software. However, limitations exist, such as the reliance on source code and the lack of support for compound predicates, which offer avenues for future research. Overall, RACING represents a promising advancement in automated RCA for fuzzing, with the potential to significantly impact the field of software security.
(Discussion) This paper explores the memory access patterns of language models (LMs), particularly focusing on the challenges they face with random access to memorized information. The authors demonstrate that while LMs can sequentially reproduce stored content effectively, they struggle with accessing information in random segments, especially in the middle of memorized sequences. To address this, the paper proposes two strategies that "recitation" (having the model first recall the content) and "permutation" (shuffling sentence order) and shows through experiments that these methods can significantly improve the model's performance. The study provides valuable insights into the limitations of memory access in LMs and suggests practical ways to enhance their performance in real-world applications. However, the research is limited to decoder-only models and does not explore larger models beyond 7 billion parameters, which could offer further insights. Additionally, experiments are conducted on a fixed-size text corpus, leaving questions about scalability to larger pretraining datasets. Despite these limitations, the paper makes an important contribution to understanding and improving memory access in language models.
(Discussion) LLMs demonstrate remarkable capabilities in various complex tasks. Recent research has delved into the possibility of enhancing the performance of these LLMs by incorporating human-annotated data. Unfortunately, this method is limited in scalability and underperforms in specific scenarios. This paper proposes a technique that can automatically generate natural language rationales using the output attribution scores that capture the influence of each feature on model predictions. The new framework, AMPLIFY, improves the prediction accuracy to about 10-25%, including those where prior approaches relying on human-annotated rationales fall short. A fundamental limitation of AMPLIFY is that it inherits the limitations of LLMs and Post hoc explanation methods.
(Discussion) The authors of the paper present a clever and straightforward approach to enhance the safety of large language models (LLMs). The idea is to have the LLM essentially double-check its own work for any potentially harmful content. When the LLM receives a prompt that could lead to a harmful response, it generates the text as usual. But then, this response is passed to another LLM that's been instructed to act as a filter. This filter LLM reads the generated text and makes a judgment call by evaluating "harmfulness". The beauty of this method lies in its simplicity - doesn't require any retraining or tweaking of the original LLM, also it doesn't involve any complex pre-processing of the input. They tested this "LLM self-defense" mechanism on two popular models, GPT-3.5 and LLaMA 2. In many cases, they were able to virtually eliminate the generation of harmful content.
(Discussion) In this paper, a new weakness in Large Language Models (LLMs) is exposed that doesn’t rely on creating special prompts, known as jail-breaking. Instead, this method, called model interrogation, uses the fact that even when an LLM refuses to answer a harmful question, the damaging reply might still be hidden in the less obvious parts of its output. By accessing the list of probable responses (top-k token predictions) available in many open-source and commercial LLMs, a person with bad intentions can manipulate the model to reveal these harmful answers. They do this by choosing less likely words from the list at certain points in the response. This technique proves to be not only different but also more effective than traditional jail-breaking, with a 92% success rate compared to 62%, and it works 10 to 20 times faster. The harmful content brought out by this method is also of higher quality. Moreover, combining model interrogation with jail-breaking methods greatly improves results over using either technique alone. The study also shows that LLMs made for specific tasks like programming can still be tricked into giving harmful responses using this method.
(Discussion) This paper introduces LLM Compiler, a novel large language model designed for code optimization and compiler tasks. Building upon the Code Llama model, LLM Compiler is specifically trained to understand intermediate representations (IR) and assembly code, showing improved performance in code generation and compiler emulation tasks. The model demonstrates strong capabilities in handling compiler-related tasks and outperforms previous models in various benchmarks. However, the paper notes some limitations. The primary issue is the model's fixed input sequence length of 16k tokens, which restricts its ability to handle very large codebases effectively. Despite efforts to split large translation units, some remain too large for the model to process. Additionally, the accuracy of the model's outputs requires rigorous evaluation and verification to ensure correctness, as any suggested optimizations should be thoroughly tested. Despite these limitations, the paper makes significant contributions to the field of compiler optimization and provides a solid foundation for future research.
(Discussion) The paper 'Unlearning Bias in Language Models by Partitioning Gradients' presents a novel technique, partitioned contrastive gradient unlearning (PCGU), to debias pretrained masked language models. PCGU selectively optimizes weights that contribute to biases by calculating gradients for contrastive sentence pairs. Evaluations using StereoSet and CrowS Pairs datasets demonstrate PCGU's effectiveness in reducing gender-profession bias with minimal impact on language modeling performance. Additionally, PCGU shows potential in mitigating biases across other domains like race and religion. However, it would have been better if there were additional experiments depending on the size of the language model. Also This technique highlights the need for ongoing research to develop robust strategies for addressing bias in language models.
(Discussion) ProPILE, a novel probing tool designed to assess the risk of Personally Identifiable Information (PII) leakage in Large Language Models (LLMs), demonstrates the feasibility of extracting PII through strategic prompt engineering. By utilizing black-box probing for data subjects and white-box probing for LLM service providers, ProPILE enables the assessment of PII leakage in models like OPT-1.3B trained on the Pile dataset. Experiments reveal that the likelihood of target PII generation increases with more specific prompts or additional linked PII details, while white-box probing with access to a limited subset of training data significantly amplifies this leakage potential. These findings underscore the effectiveness of ProPILE as an assessment tool and highlight the need for ongoing research and robust mitigation strategies to address privacy vulnerabilities in LLMs.
(Discussion) This paper proposes a bi-directional transformer-based guessing framework, referred to as PassBERT, which applies the pre-training/fine-tuning paradigm to password guessing attacks. First, a model pre-trained on the general password distribution was prepared, which was then fine-tuned on three specifically designed attack approaches. These methods reflect real-world attack scenarios and include: 1) conditional password guessing, which recovers the complete password given a partial one; 2) targeted password guessing, which compromises the password of a specific user using personal information; and 3) adaptive rule-based password guessing, which selects rules to generate rule-transformed password candidates. The experimental results show that the fine-tuned models can outperform state-of-the-art models by 14.53%, 21.82%, and 4.86% in the three attacks, respectively, demonstrating the effectiveness of bi-directional transformers on downstream guessing attacks. Furthermore, a hybrid password strength meter was proposed to mitigate the risks from these three types of attacks.
(Discussion) This paper introduces an innovative multi-stage ranking system for extracting and annotating adversarial techniques from threat intelligence reports, leveraging language models fine-tuned for cybersecurity tasks. The system demonstrates significant improvements in accuracy and recall compared to previous methods, with enhanced performance on verbose datasets like MITRE ATT&CK Reports and WeLiveSecurity. The comprehensive approach, including testing across various datasets and the introduction of a public dataset with 6.6K annotations, strengthens the study's contributions to the field. However, the observed performance disparity across datasets highlights the need for further refinement to handle concise descriptions more effectively. Overall, the study advances automated threat technique annotation and provides a solid foundation for future cybersecurity research and development.
(Discussion) Detecting anomalies within system logs is paramount to protecting the system from an attack or malfunction. Traditional methods employ regular expressions or machine learning models to identify anomalous events. These approaches depend on handcrafted features and are unable to capture temporal information. For example, malicious logs could be benign on their own, but the collection of these logs could be malicious. Recently, deep learning models such as recurrent neural networks (RNNs) have been widely used to evaluate these sequences and can capture temporal information. However, RNNs cannot encode contextual information in a bi-directional manner. This paper proposes a log anomaly detection model based on BERT. BERT can capture contextual information in a bi-directional manner making it suitable for this task. The model is evaluated on three datasets: Hadoop Distributed File System (HDFS), BlueGene/L Supercomputer System (BGL), and Thunderbird-mini dataset. The baseline models are Principal Component Analysis (PCA), One-Class SVM (OCSVM), IsolationForest (iForest), LogCluster, DeepLog, and LogAnomaly. Their proposed model demonstrates superior performance over all baseline state-of-the-art models in all datasets. However, the detailed process in training is not described. For example, BERT splits the training step into a pretraining and finetuning phase. However, LogBERT seems to only discuss the pretraining phase and have no mention about the fine tuning phase. As a result, it is not clear if the fine tuning phase is excluded or it is not mentioned.
(Discussion) This paper proposes an attack method that causes aligned language models to generate undesirable behaviors. Aligned language models refuse to respond to harmful queries. To enable these language models to respond to harmful queries, this paper attaches a suffix to the query. This approach makes the LLM generate a positive response instead of refusing to answer. Positive responses to harmful queries were obtained from ChatGPT, Bard, Claude, LLaMA-2-Chat, Pythia, and Falcon, with a much higher success rate for GPT-based models. The evaluation section only demonstrates the high success rate of the proposed attack method. It would have been better if the paper included the limitations of this study and a model ablation study section.
(Discussion) The paper titled 'Membership Inference via Backdooring' presents a novel approach to addressing data privacy concerns in machine learning by introducing a method called Membership Inference via Backdooring (MIB). The main contribution of MIB lies in its ability to mark a small number of data samples, which, when used by an unauthorized party to train a model, allows the data owner to later identify this misuse through black-box queries and statistical hypothesis testing. However, several limitations and challenges accompany this promising approach. Firstly, the success of MIB hinges on the ability to inject backdoor triggers in a manner that remains undetectable by the unauthorized party. While the paper discusses techniques to make the triggers imperceptible, there is an inherent risk that sophisticated adversaries could develop detection methods to identify and neutralize these backdoors. Furthermore, the approach's reliance on statistical hypothesis testing to provide guarantees for inference results, while innovative, may still be vulnerable to model-specific variations and adversarial defenses. However, further exploration is needed to assess the robustness of MIB across various types of datasets and models not covered in the experiments of the paper. Additionally, it is regrettable that the examples from the various datasets experimented in Figure 5 were not also included.
(Discussion) The paper titled 'Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations' introduces Llama Guard, a model developed to enhance safety in human-AI interactions. The model leverages the Llama2-7b architecture, fine-tuned to classify and mitigate safety risks in both user prompts and AI responses. The core contribution is a comprehensive safety risk taxonomy that guides the classification process, encompassing categories like violence, hate speech, sexual content, and self-harm. Llama Guard outperforms existing moderation tools on benchmarks such as the OpenAI Moderation Evaluation dataset and ToxicChat. The model supports customizable and adaptive use through zero-shot and few-shot learning, and its weights are publicly available for further research and development. However, the work has some limitations. Llama Guard's common sense knowledge is constrained by its training data, which can lead to incorrect judgments. Its performance in non-English languages is not guaranteed, and the quality of fine-tuning labels may not comprehensively cover all policy aspects, leading to subpar performance in some cases.
(Discussion) This study collects and analyzes data from human participants through malware classification games, clearly revealing the differences between human and machine learning models. This is an original approach not seen in previous studies. However, the number of samples used in the experiment is limited to 20, which may be somewhat insufficient to represent the overall malware classification problem. Nevertheless, their results provide practical insights that can be applied directly to training malware analysis experts and improving ML models. For example, they propose integrating dynamic behavior analysis of human experts into ML models. This work provides important insights into how human experts and ML models classify malware, and suggests that a hybrid approach that incorporates human intuition and experience into ML models could be highly effective in future malware defense.
(Discussion) Let there be a video of a vehicle moving backward. An anomaly detection by frame alone would fail to capture any anomalies; instead, the model requires a sequence of frames to identify the anomaly. This paper proposes a Transformer-based anomaly detection in aerial videos from an Unmanned Aerial Vehicles (UAVs). By leveraging the Transformer encoder, they claim that the model can preserve spatiotemporal information. The main contribution comprises an annotated dataset for realistic anomalous events, a benchmark for anomaly detection, and a baseline model ANDT. ANDT exhibits the best performance for certain scenes but does not do so for all scenes tested in the paper. Initial claims suggest that previous models do not preserve spatiotemporal information. However, previous models demonstrate decent performance on the provided dataset. In Figure 4, the MemAE model can infer the vehicle moving backward as an anomalous event, demonstrating its ability to preserve spatiotemporal information. Furthermore, the train test split in Table 1 is odd yet there is no mention why it was done in this way. For example, the test set is larger than the train set in the Bike roundabout scene.
(Discussion) In many ways, the compilation process is destructive. High-level constructs, variable names, and comments present in the source code are often lost. Nevertheless, decompilers aim to retrieve the source from a binary. Decompilation results vary between decompilers. So, what is a good decompilation? One could argue that fewer goto instructions indicate good decompilation. Because multiple gotos signify the decompiler failed to structure the control flow. This paper argues that good decompilation is similar to the source code and points out that 3,754 goto instructions are present in the Linux kernel. Since a goto instruction could be intentional by the developer, they argue we should separate intended gotos from unintended gotos. The authors investigate the gcc compiler to identify the transformation passes that cause unintended gotos. Afterward, they propose a structuring algorithm that deoptimizes code to remove unintended gotos while preserving intended ones. SAILR, the proposed structuring algorithm, is implemented on the angr decompiler. SAILR demonstrates decent performence against state-of-the-art decompilers. The paper evaluates SAILR with 7,355 functions from 26 popular Debian packages. However, the number of functions is oddly low, considering the number of packages used.
(Discussion) This paper proposes the Membership Inference method 'MIN-K% PROB' in LLM(Large Language Model). The problem presented in this paper is the challenge of detecting pretraining data. This is because LLM developers do not open the data used for pertaining, and since pretraining involves training on data instances once at a time, it makes detection even more difficult. Hence, this paper assumes that the Membership Inference Detector cannot know the distribution of pretraining data. This means that there is no Reference Model (e.g., Shadow Model) used in Membership Inference techniques. Consequently, this paper proposes the Reference-free Method, MIN-K% PROB. The 'Min-K% PROB' method selects outlier tokens with low token probability to create a set, and then uses the average of this set as the threshold for inferring between members and non-members. This method seems to be efficient as it infers membership based on the token-level probability during the pretraining. Additionally, using this method as an evaluation for Unlearning techniques, as demonstrated in the case study, would be beneficial.
(Discussion) The paper investigates the application of prompt learning with Large Language Models (LLMs) such as GPT-3 and T5 to address the issue of toxic online content. It focuses on three primary tasks: toxicity classification, toxic span detection, and detoxification, showing that prompt learning can perform as well as or even better than traditional models specifically trained for these tasks. Notably, this approach achieves a 10% improvement in toxicity classification and significantly lowers toxicity scores in detoxification tasks while maintaining the semantic integrity of the content. The study is distinguished by its innovative use of prompt tuning for managing toxic content and its thorough evaluation across five model architectures and eight different datasets, which verifies the method's effectiveness and efficiency. However, despite these strengths, the approach's dependency on the quality of datasets and its ability to generalize to unseen toxic content remain potential weaknesses. Furthermore, the complexity involved in designing effective prompts and the possible misuse of the techniques are also concerns.
(Discussion) This paper proposes 'ANUBIS', a provenance graph-based supervised APT detection framework. ANUBIS is a method leveraging a BNN(Bayesian Neural Network) to overcome limitations in current APT detection methods using provenance graphs. The authors hypothesize that an attacker cannot breach the probability graph used to train ANUBIS, nor does it violate the event logging mechanism of the host system. These assumptions provide a strong premise for security scenarios, but they have limitations that do not take into account the practical considerations. However, this paper is significant because it has demonstrated that it is effective in predicting the nature of the activity by introducing a new graph neighbor encoding method.
(Discussion) This paper is a measurement study on the IoT Malware Lifecycle. It collects around 166K Linux-based IoT malware samples collected over a year to measure the characteristics of IoT Malware. The paper concludes with some characteristics that differentiate IoT malware from traditional malware, such as most IoT malware being a variant of the Mirai botnet. However, there does not seem to be unexpected findings. A slight complaint is that figures within the paper had unclear captions. Although the paper explains the concepts, some figures are difficult to understand.
(Discussion) In this paper, a dataset composition for testing the latest BCSD (Binary Code Similarity Detection) techniques is proposed, and analysis of test results for various BCSD models, including the latest GNN-based BCSD and Embedding-based BCSD, has been performed. However, in the evaluation section, it is necessary to divide the results for similarity and similarity lacking into separate tables to enhance clarity. Additionally, there is a lack of explanation for the many abbreviations used in the dataset composition, making it difficult to understand.
(Discussion) The paper discussion focuses on an innovative approach to optimizing the reward function of a Reinforcement Learning (RL) model to mitigate undesired behaviors. The authors detail a three-stage algorithm comprising exploration, quantization, and learning, each critical to the model's development. Notably, the paper demonstrates the model's effectiveness through three distinct evaluations, addressing the reduction of toxicity, improvement over negative baselines, and minimization of repetitive actions. This structured evaluation underscores the model's capability to unlearn specific unwanted behaviors, which outperforms both baselines and state-of-the-art RL methods.
In this paper, the focus is on defining three undesirable behaviors and proposing three methods (e.g. reward function) to forget each behavior. It seems inefficient to train a model separately for each behavior.
(Discussion) This paper designs a multidimensional detection framework to detect lateral movement behavior against APT attacks in an intranet environment based on the SMB protocol. However, contrary to this purpose, the experimental results are unfortunate in that the purpose and results differ because they are about how well malware has been classified.
(Discussion) Systems generate logs to record internal events. These logs are used after a cyber incident for forensic investigation. Unfortunately, the system generates numerous logs during its runtime and majority of the analysis is manual. This paper proposes a deep autoencoder for anomaly detection to assist analyzers by highlighting anomalous events in the Linux kernal system logs.
Anomaly detection is a common application for autoencoders. It is utilized to analyze network traffic, logs, etc. It is difficult to identify what is different from previous work and this paper. Furthermore, the proposed model is evaluated with old dataset against ML methods such as SVM. It would have been preferable to see the model’s performance on up-to-date dataset against state-of-the-art models.
(Discussion) There are efficient ways to represent binary functions using a tokenizer (e.g., BPE) in assembly language. I'm curious about why the suggestion is to divide them into seven features like Vex-IR instructions, LibcCalls, Constants, etc., and create specific tokenizers for each feature to generate one-hot vectors. I also wonder if this approach is genuinely effective. The explanation about whether this method of tokenization has a positive impact on BCSD (Binary Code Similarity Detection) performance seems insufficient.
(Discussion) The paper utilizes adversarial examples to retain the original decision boundary during unlearning. This approach is intriguing yet it raises questions as to what it means to “forget” a data point. The paper states that misclassification signifies forgetting yet “Right to be Forgotten” seems to imply the removal of a certain training data.
Malware datasets inevitably contain incorrect labels due to the shortage of expertise and experience needed for sample labeling. Previous research demonstrated that a training dataset with incorrectly labeled samples would result in inaccurate model learning. To address this problem, researchers have proposed various noise learning methods to offset the impact of incorrectly labeled samples, and in image recognition and text mining applications, these methods demonstrated great success. In this work, we apply both representative and state-of-the-art noise learning methods to real-world malware classification tasks. We surprisingly observe that none of the existing methods could minimize incorrect labels’ impact. Through a carefully designed experiment, we discover that the inefficacy mainly results from extreme data imbalance and the high percentage of incorrectly labeled data samples. As such, we further propose a new noise learning method and name it after MORSE. Unlike existing methods, MORSE customizes and extends a state-of-the-art semi-supervised learning technique. It takes possibly incorrectly labeled data as unlabeled data and thus avoids their potential negative impact on model learning. In MORSE, we also integrate a sample re-weighting method that balances the training data usage in the model learning and thus handles the data imbalance challenge. We evaluate MORSE on both our synthesized and real-world datasets. We show that MORSE could significantly outperform existing noise learning methods and minimize the impact of incorrectly labeled data.
(Abstract) Pre-trained models for Natural Languages (NL) like BERT and GPT have been recently shown to transfer well to Programming Languages (PL) and largely benefit a broad set of code-related tasks. Despite their success, most current methods either rely on an encoder-only (or decoder-only) pre-training that is suboptimal for generation (resp. understanding) tasks or process the code snippet in the same way as NL, neglecting the special characteristics of PL such as token types. We present CodeT5, a unified pre-trained encoder-decoder Transformer model that better leverages the code semantics conveyed from the developer-assigned identifiers. Our model employs a unified framework to seamlessly support both code understanding and generation tasks and allows for multi-task learning. Besides, we propose a novel identifier-aware pre-training task that enables the model to distinguish which code tokens are identifiers and to recover them when they are masked. Furthermore, we propose to exploit the user-written code comments with a bimodal dual generation task for better NL-PL alignment. Comprehensive experiments show that CodeT5 significantly outperforms prior methods on understanding tasks such as code defect detection and clone detection, and generation tasks across various directions including PL-NL, NL-PL, and PL-PL. Further analysis reveals that our model can better capture semantic information from code. Our code and pre-trained models are released at https: //github.com/salesforce/CodeT5.
(Abstract) Code is seldom written in a single left-to-right pass and is instead repeatedly edited and refined. We introduce INCODER, a unified generative model that can perform program synthesis (via left-to-right generation) as well as editing (via masking and infilling). InCoder is trained to generate code files from a large corpus of permissively licensed code, where regions of code have been randomly masked and moved to the end of each file, allowing code infilling with bidirectional context. Our model is the first large generative code model that is able to infill arbitrary regions of code, which we evaluate in a zero-shot setting on challenging tasks such as type inference, comment generation, and variable re-naming. We find that the ability to condition on bidirectional context substantially improves performance on these tasks, while still performing comparably on standard program synthesis benchmarks in comparison to left-to-right only models pretrained at similar scale. Our models and code are publicly released.
(Abstract) Program synthesis or code generation aims to generate a program that satisfies a problem specification. Recent approaches using large-scale pretrained language models (LMs) have shown promising results, yet they have some critical limitations. In particular, they often follow a standard supervised fine-tuning procedure to train a code generation model only from the pairs of natural-language problem descriptions and ground-truth programs. Such paradigm largely ignores some important but potentially useful signals in the problem specification such as unit tests, which thus often results in poor performance when solving complex unseen coding tasks. To address the limitations, we propose “CodeRL”, a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning (RL). Specifically, during training, we treat the code-generating LM as an actor network, and introduce a critic network that is trained to predict the functional correctness of generated programs and provide dense feedback signals to the actor. During inference, we introduce a new generation procedure with a critical sampling strategy that allows a model to automatically regenerate programs based on feedback from example unit tests and critic scores. For the model backbones, we extended the encoder-decoder architecture of CodeT5 with enhanced learning objectives, larger model sizes, and better pretraining data. Our method not only achieves new SOTA results on the challenging APPS benchmark, but also shows strong zero-shot transfer capability with new SOTA results on the simpler MBPP benchmark.
(Abstract) Modern disassembly tools often rely on empirical evaluations to validate their performance and discover their limitations, thus promoting long-term evolvement. To support the empirical evaluation, a foundation is the right approach to collect the ground truth knowledge. However, there has been no unanimous agreement on the approach we should use. Most users pick an approach based on their experience or will, regardless of the properties that the approach presents. In this paper, we perform a study on the approaches to building the ground truth for binary disassembly, aiming to shed light on the right way for the future. We first provide a taxonomy of the approaches used by past research, which unveils five major mechanisms behind those approaches. Following the taxonomy, we summarize the properties of the five mechanisms from two perspectives: (i) the coverage and precision of the ground truth produced by the mechanisms and (ii) the applicable scope of the mechanisms (e.g., what disassembly tasks and what types of binaries are supported). The summarization, accompanied by quantitative evaluations, illustrates that many mechanisms are ill-suited to support the generation of disassembly ground truth. The mechanism best serving today’s need is to trace the compiling process of the target binaries to collect the ground truth information. Observing that the existing tool to trace the compiling process can still miss ground truth results and can only handle x86/x64 binaries, we extend the tool to avoid overlooking those results and support ARM32/AArch64/MIPS32/MIPS64 binaries. We envision that our extension will make the tool a better foundation to enable universal, standard ground truth for binary disassembly.
2025
(Discussion) Recent studies have shown that Large Language Model (LLM)-based agents are vulnerable to security risks; that is, malicious prompts can exploit sensitive operations. To address this, the paper introduces AgentFuzz, a greybox fuzzing framework designed to detect taint-style vulnerabilities in LLM-based agents. AgentFuzz leverages dataflow analysis to identify predefined sinks, such as SQL and code injection, and constructs call chains that guide LLM-assisted prompt generation, forming the main seed pool. Afterward, each seed's execution trace is evaluated at both semantic and distance levels to schedule the seed to mutate first. Using this approach, AgentFuzz uncovered 34 previously unknown high-risk vulnerabilities, 23 of which were assigned CVE at the time of publication. This work is a novel direction for identifying vulnerabilities in LLM-based agents. (Jiyong).
(Discussion) This paper proposes a hybrid framework that combines Code Property Graphs (CPGs) with Large Language Models (LLMs) to improve software vulnerability detection. By constructing vulnerability-focused code slices, the system enables LLMs to capture security-critical context across both function-level and project-level codebases. Empirical evaluation demonstrates improvements over state-of-the-art baselines, strong robustness against code transformations, and effective generalization to unseen datasets. However, limitations remain, including the inability of CPG-based methods to capture runtime-dependent vulnerabilities, the scarcity of high-quality datasets, and context length constraints in the fine-tuned LLMs. In addition, the system’s performance degrades on deeply nested code structures and shows reduced effectiveness on underrepresented CWEs (Shakhzod).
(Discussion) This paper introduces EvoCrawl, a novel web application crawler that leverages evolutionary search to overcome the limitations of traditional state exploration in vulnerability detection. Its key strengths lie in the innovative use of dependency tracking and a fitness-based evolutionary algorithm, which enable it to efficiently navigate complex interaction sequences, achieve significantly higher code coverage, and uncover real zero-day vulnerabilities across widely used platforms. The implementation is technically robust, integrating multiple modules (ESM, PM, IVD, XVD) and demonstrating practical effectiveness through extensive case studies. Nonetheless, the work has some shortcomings: the design section is at times confusing in its presentation, terminology inconsistencies appear, and the system currently requires manual parameter tuning while struggling with certain token-handling scenarios. Despite these limitations, EvoCrawl represents a meaningful advancement in automated security testing by showing how evolutionary algorithms can push the boundaries of web application exploration and vulnerability discovery. (Suyeon)
(Discussion) This paper conducts the first empirical study of Rust-for-Linux, revealing both its successes and compromises. Its strengths lie in a thorough analysis of drivers, commits, and benchmarks, showing that Rust improves code quality, testing, and developer engagement while making Linux more securable. However, unsafe code remains inevitable, performance is inconsistent across drivers, and the steep learning curve complicates adoption. Despite these limitations, the study marks an important step in understanding how Rust reshapes kernel development, offering valuable insights for balancing safety, efficiency, and community growth in future RFL work. (Yiyue)
(Discussion) KernelGPT takes a stepwise approach to interpreting kernel driver and socket code with an LLM—recovering identifiers, types, and dependencies—and iteratively verifying and repairing its outputs to produce syzlang specifications, thereby extending fuzzing into paths that earlier tools left uncovered. In evaluation, it generated 532 new syscall specifications and 294 new type definitions; combined with Syzkaller, these additions unlocked 20,472 additional unique basic blocks and surpassed state-of-the-art baselines in both coverage and crash discovery. This specification lift translated into practice: the system uncovered 24 new kernel bugs (21 confirmed, 11 assigned CVEs, 12 already patched), underscoring a clear link between specification quality and fuzzing effectiveness. Still, the current pipeline relies largely on syntactic and shallow semantic checks and can miss runtime semantic errors, while risks such as LLM hallucination and version drift remain.
(Discussion) This paper analyzed and discovered vulnerabilities in email system attachment detection. It demonstrated that malicious code detection can be evaded by exploiting differences in MIME parsing methods between email security detectors and clients. Using this approach, they developed a framework and discovered a total of 24 vulnerabilities across 7 popular email clients. Although there are limitations such as potential errors in many real-world cases due to reliance on open-source parser filtering, and the approach is only applicable within restricted scenarios, they achieved significant results including being the first to analyze MIME parsing ambiguity and actually discovering vulnerabilities.(Hyoungjun)
(Discussion) This paper presents Coda, a neural-based framework for program decompilation that departs from traditional rule-based approaches by employing a two-phase design: code sketch generation using an instruction type-aware encoder and AST decoder, followed by iterative error correction guided by ensembled error predictors and compiler feedback. The framework achieves significantly higher program recovery accuracy than both conventional decompilers and Seq2Seq-based models, demonstrating its ability to preserve semantics as well as functionality. However, its effectiveness is constrained by assumptions such as access to compiler configurations and unoptimized binaries (-O0), and performance drops on more complex ISAs like x86-64 or longer programs due to input length and architectural challenges. While the study convincingly shows the feasibility of end-to-end neural decompilation and raises concerns about software IP protection, its applicability to real-world scenarios with optimized binaries, diverse architectures, and richer data structures remains an open challenge. (Mijin)
(Discussion) The study explores how OAuth 2.0, when applied in integration platforms like workflow automation, virtual assistants, and smart homes, can be exploited due to a role reversal in trust relationships. The authors identify two novel cross-app attacks, Cross-app OAuth Account Takeover (COAT) and Cross-app OAuth Request Forgery (CORF) that allow malicious apps to hijack or forge account linkages, sometimes with just a single click. A strength of this work is that it introduces a new threat model and attack types while keeping the scenarios realistic, and the authors also discovered real bugs in widely used platforms, even leading to bug bounty rewards. On the other hand, all attacks rely on the assumption that a victim connects to a malicious app, which is a relatively strong premise, and some of the proposed defenses are challenging to put into practice. Still, the contribution is meaningful in showing the prevalence of these vulnerabilities across major platforms such as Microsoft, Google, and Amazon, and in providing systematic analysis and concrete guidance for securing the ecosystem. (Suyeon)
(Discussion) This paper presents a forensic framework for investigating conversational AI services—specifically ChatGPT, Gemini, Copilot, and Claude—by treating them as forensic targets rather than tools. It analyzes how these services store and handle data across desktop apps, mobile apps, and web browsers, identifying various artifacts such as chat logs, cache files, attachments, and account information. The authors propose a five-phase forensic process (Preparation, Identification, Collection, Analysis, and Reporting) and provide open-source tools to support data extraction and normalization. Through detailed technical analysis and two case studies—a ransomware investigation and a copyright dispute—the paper demonstrates how forensic investigators can uncover evidence even from deleted or inaccessible accounts. However, the study has limitations: it relies heavily on platform-specific behaviors that may change over time and cannot recover deleted data without access credentials or device-level traces. Additionally, the specific forensic platform used is not clearly stated, and key assumptions for the investigation process are not explicitly defined. (Shakhzod)
(Discussion) This paper presents LLM4Decompile, an open-source series (1.3B–33B) tailored for binary-to-C decompilation via two complementary pipelines—an end-to-end model trained on disassembled ASM and a refinement model that cleans Ghidra output—together with pragmatic training choices (O0–O3 augmentation, duplicate filtering, and a two-stage “compilable→executable” curriculum). Its chief strengths are careful experimental design (per-optimization analysis, ablations disentangling compilable vs. executable distributions), clear, execution-based metrics, and a convincing demonstration that coupling LLMs with classical tooling (Ghidra+Ref) lifts re-executability beyond both GPT-4o and Ghidra alone; the obfuscation study is also a thoughtful, policy-aware addition. However, several weaknesses temper the claims: (i) external validity—the main targets are single functions or small C snippets on x86; results may not transfer to multi-file projects, other ISAs, or other languages; (ii) the key metric (re-executability under unit tests) can overestimate semantic recovery and underreport subtle miscompilations, while the GPT-based readability score introduces subjectivity; (iii) the “end-to-end” path still relies on disassembly (Objdump), sidestepping raw-byte modeling challenges; (iv) the refinement data decompiled from non-executable objects may diverge from real analyst workflows; and (v) scaling conclusions are confounded by the undertrained 33B model and limited compute, with no detailed error taxonomy or human-in-the-loop studies to gauge usefulness in practice. Overall, the work is a substantial step toward LLM-assisted decompilation and a strong empirical baseline, but broader architectures, languages, and user-centered evaluations will be essential to cement its impact. (Minseok)
(Discussion) This paper introduces JiT, an information-theoretic framework for zero-shot machine unlearning, addressing the challenge of removing specific training data from models without access to retained data. The authors propose a novel method that estimates a sample’s influence through local information gain, measured by the curvature of the model’s output space. By minimizing gradients in the neighborhood of forget samples, JiT effectively erases their impact while preserving model performance. The method outperforms prior zero-shot techniques across full-class, sub-class, and random unlearning benchmarks, achieving comparable performance to retraining with significantly lower computational cost. For instance, JiT maintains high accuracy on retained data and achieves similar entropy distributions to retrained models on CIFAR-10. However, the approach has notable limitations, including sensitivity to hyperparameter tuning, incompatibility with batch normalization, and lack of formal unlearning certification. JiT marks a significant advance in practical zero-shot unlearning, offering a scalable and effective method under strict constraints, yet highlights ongoing challenges in reliable data removal from black-box models.(Yiyue)
(Discussion) This paper proposes URL Inspection Tasks, an active defense mechanism to reduce phishing success in emails by requiring users to engage with a URL before visiting it. Inspired by CAPTCHAs, the system presents one of three tasks—clicking the correct domain, highlighting it, or re-typing it—designed to focus user attention on the domain and verify alignment with their intent. In a large-scale online study (2,673 participants), these tasks cut phishing success from 74.5% (control) to 35%, with particularly strong performance against typosquatting (down to 17.1%). However, the approach has limitations, including minor increases in false positives, added user burden (7–10 seconds), and reduced effectiveness for users unfamiliar with URL structures or non-Latin scripts. Despite these trade-offs, the work demonstrates that lightweight, interactive tasks can meaningfully reduce phishing risks by enhancing user awareness.(Jiyong)
(Discussion) This paper presents a security-focused analysis of Google's Agent-to-Agent (A2A) protocol, a foundational framework for enabling secure communication in agentic AI systems. As autonomous agents become more complex and interconnected, traditional security models fall short. The authors apply the MAESTRO threat modeling framework—tailored for multi-agent AI—to identify key vulnerabilities in A2A systems, including Agent Card spoofing, task replay attacks, and prompt injection via poisoned metadata. Through two detailed case studies, the paper illustrates real-world risks and proposes robust mitigation strategies such as strict schema validation, mutual TLS, digital signatures, and secure Agent Card discovery. While offering actionable guidance and best practices for secure implementation, the work's effectiveness relies heavily on disciplined protocol adherence and may face scalability challenges in untrusted ecosystems. Nonetheless, it marks a critical step in building secure, trustworthy infrastructure for the next generation of interoperable agentic applications. (Yujeong)
(Discussion) This paper introduces the Model Context Protocol (MCP), a standardized framework that streamlines interaction between AI models and external tools, addressing long-standing inefficiencies in tool integration, manual API wiring, and fragmented plugin systems. Inspired by the Language Server Protocol, MCP allows AI agents to dynamically discover and invoke external tools through a unified interface, enhancing autonomy and flexibility. The paper presents a comprehensive overview of the MCP ecosystem, detailing its architecture—comprising MCP hosts, clients, and servers—as well as the protocol's workflow and server lifecycle phases: creation, operation, and update. Notably, the authors conduct the first systematic analysis of MCP's security landscape, identifying risks such as name collision, installer spoofing, sandbox escapes, and privilege persistence. With real-world adoption by industry leaders like OpenAI, Cloudflare, and Anthropic, MCP is rapidly becoming foundational for AI-native applications. However, its ecosystem remains immature and decentralized, lacking formal security standards, version control, and authentication protocols. The paper emphasizes the urgent need for centralized package management, secure installation frameworks, and improved debugging tools to support MCP's safe and scalable growth. While MCP marks a significant step toward enabling agentic AI workflows, its current vulnerabilities underscore the importance of proactive governance, rigorous security audits, and continued research into robust deployment mechanisms. (Yujeong)
(Discussion) This paper presents TYPEPULSE, a static analysis tool for detecting type confusion bugs in Rust programs, addressing a critical gap in memory safety analysis for Rust's unsafe code. Unlike C/C++, Rust enforces safety through strict compile-time checks, but unsafe pointer conversions can still lead to severe bugs. TYPEPULSE targets three key bug types—misalignment, inconsistent layout, and mismatched scope—by analyzing type conversions, trait-bound generics, and pointer aliasing. It builds a property graph to capture interprocedural behaviors and uses this to identify invalid type accesses. Evaluated on 3,000 top Rust packages, TYPEPULSE uncovered 71 previously unknown bugs, surpassing the number reported to RustSec over the last five years. However, the tool's precision (75.5%) is limited by its reliance on static patterns and its difficulty in interpreting complex developer-enforced checks. Despite these limitations, TYPEPULSE represents a major step forward in automated detection of unsafe Rust bugs, offering both greater coverage and accuracy than existing tools like Clippy and Rudra. (Hyoungjun)
(Discussion) This paper analyzes Korea Security Applications (KSA) 2.0, mandatory security software for South Korean financial services, revealing that government-mandated security solutions can create more vulnerabilities than they prevent. The researchers identified four fundamental design flaws and 19 concrete vulnerabilities, including the paradoxical finding that anti-keylogging software could be exploited for keylogging attacks. With 97% of banking users having installed KSA despite 59% not understanding its functions, the study demonstrates how mandatory deployment amplifies security risks across national digital infrastructure. The vulnerabilities were actively exploited in North Korean cyberattacks in 2023, showing how mandatory software becomes attractive targets for nation-state actors. The remediation process revealed systemic issues where vendors resist fundamental security fixes due to backward compatibility concerns, creating perverse incentives that prioritize stability over security. The study serves as a cautionary tale for policymakers, emphasizing the need for rigorous security assessment before mandating any security technology. (Mijin)
(Discussion) This paper presents XDAC, a framework designed to detect and attribute LLM-generated comments in Korean news platforms, addressing the critical challenge of maintaining online discourse integrity in an era of sophisticated AI-generated content. The research tackles a significant gap in existing detection methods, which are inadequate for short-form content (Korean news comments average 51 characters versus GPTZero's 250-character minimum requirement). Using XAI-driven linguistic analysis, the authors identified distinguishing patterns between human and LLM content, such as LLMs' preference for formal structures versus humans' use of informal expressions and repeated characters. The framework achieves impressive performance with 98.5% F1 score in detection and 84.3% in attribution, successfully identifying 27,029 potential LLM-generated comments from 5.24M real-world comments on Naver. However, the approach faces significant limitations including reliance on artificially generated training data, heavy dependence on Korean-specific linguistic patterns that limit cross-lingual generalizability, and concerning vulnerabilities to adversarial attacks. The research represents a significant advancement in short-form content detection but highlights the ongoing arms race between generation and detection technologies, emphasizing the need for more robust defenses.(Mijin)
(Discussion) This paper proposes 'VulSim', a novel framework to solve the persistent problem of data scarcity in vulnerability detection. Instead of simply increasing data volume, VulSim enhances informational depth by integrating a code's semantic, contextual, and syntactic properties and analyzing the similarity of neighboring data. This unique approach surpassed existing state-of-the-art models with 75.41% accuracy on the Microsoft CodexGLUE benchmark and demonstrated excellent generalizability and practical potential by achieving a high recall of 85% on a completely new dataset. Despite these achievements, several clear limitations exist. First, as the model was validated only on C code, its effectiveness on the unique vulnerability patterns of other programming languages remains unknown. Second, its function-level analysis carries a methodological constraint, as it may miss complex vulnerabilities that span multiple functions or be diluted by irrelevant code. Finally, from a practical standpoint, the trade-off of sacrificing precision (53% on unseen data) for high recall can become a major obstacle to adoption, as it would incur significant costs in triaging false positives in a real-world environment. In conclusion, while VulSim presents an effective alternative beyond data-centric approaches, further research is needed to fully realize its potential. Future work must go beyond expansion to other languages and focus on resolving granularity issues and optimizing the precision-recall balance to enhance its reliability and efficiency in real-world development settings. (Intae)
(Discussion) This paper introduces AudioMarkNet, a proactive watermarking technique designed to embed imperceptible watermarks into real speech. Its goal is to prevent the misuse of genuine speech data for speaker adaptation in text-to-speech (TTS) systems—one of the main techniques used to generate realistic deepfake audio. Unlike traditional methods that detect deepfakes after they’re created, AudioMarkNet works preventively. The authors show that their method effectively detects fake speech from both open-source and commercial TTS models and remains robust even under non-adaptive and adaptive attacks. Code and examples are publicly available. Although the method is effective in detecting cloned voices, a limitation is that the watermarks may become noticeable during silent sections of the audio. This can allow adversaries to identify and potentially filter out watermarked speech (Shakhzod).
(Discussion) This paper introduces a novel technique called LLMmap for fingerprinting LLMs integrated into applications. Unlike previous methods that require access to internal model parameters or rely on watermarking, LLMmap uses an active querying strategy: it sends carefully crafted prompts to an application and analyzes the model's responses to identify its specific version. The authors demonstrate that LLMmap can accurately identify 42 popular LLM versions with over 95% accuracy using as few as 8 queries. The study also shows how fingerprinting enables more precise adversarial attacks and discusses the difficulty of building effective defenses, as LLM outputs are often distinct and hard to obfuscate consistently without degrading utility. However, the study has key limitations: it focuses solely on fingerprinting model outputs in interactive settings, while future work could extend to fingerprinting LLM-generated documents, images, or videos. Additionally, its approach to handling novel models is limited, new LLM versions are merely categorized as "unseen" without finer granularity to identify their exact variant.
(Discussion) The paper presents a large scale analysis of Telegram’s conspiracy theory ecosystem by combining a novel Conspiracy Resource Dataset drawn from prior academic work on YouTube, Reddit, and other fringe platforms with the TGDataset of 120,000 public Telegram channels. Using URL matching and community detection on a forward graph of 498 million messages, the authors identify four distinct "conspiracy communities" totaling nearly 18,000 channels, then reveal how these channels monetize their audiences through donations, crowdfunding, and affiliate links raising almost $71 million from over 900,000 backers. They also release two datasets and prototype mitigation tools (a Telegram bot and browser plug-in) to warn users about suspicious channels and links. However, the study’s reliance on link-based detection overlooks monetization embedded in media or private features (e.g., sponsored images, webinars, Telegram’s own Sponsored Messages), its Conspiracy Resource Dataset may be biased toward English language sources and known platforms (potentially under-detecting non-English communities), and the 9 to 15 month gap between message collection and monetization crawling could understate ongoing campaigns. Furthermore, using GPT-4o to classify promotional versus discrediting messages, while efficient, introduces misclassification risk, and the analysis stops short of firmly linking channel operators to the funds they promote issues that call for more comprehensive signal integration, finer grained validation, and evaluation of real-world efficacy.
(Discussion) Identifying vulnerabilities early is crucial for preventing security incidents. Moreover, the common weakness enumerations (CWEs) provide a useful taxonomy for organizing various vulnerabilities in code. Building on CWEs, this paper propose a hierarchical contrastive learning framework to classify code vulnerability types by clustering related weaknesses in a shared embedding space. While the approach shows promising results and outperforms existing baselines on their selected dataset, the evaluation raises few concerns. The model is only tested on a curated vulnerability dataset (ie., BigVul and PrimeVule), which, despite including non-vulnerable functions, may not reflect the complexity of large scale projects like the Linux kernel. Moreover, the model performs worse on PrimeVul -- a dataset with higher label accuracy -- suggesting limitations in a more realitic scenario. Although, the framwork demonstrates higher performance against state-of-the-art models, its application remains uncertain without further investigation.
(Discussion) TokenFormer introduces a novel approach by integrating a p-attention layer into a Transformer-based architecture. Unlike the standard self-attention mechanism, the p-attention layer stores its weights as external parameters. This design allows for more efficient fine-tuning on new datasets with reduced computational overhead. A key distinction of this method is that both the original attention weights "old parameters" and the newly introduced weights "new parameters" remain trainable during fine-tuning. This contrasts with many existing fine-tuning strategies, which typically freeze the original parameters to prevent catastrophic forgetting. Interestingly, the authors do not explicitly address catastrophic forgetting in the paper, despite deviating from the conventional practice of parameter freezing.
(Discussion) This paper tackles the challenge of factual unlearning in LLMs by proposing AltPO, a method that integrates negative feedback with in-distribution positive feedback to improve coherence and avoid nonsensical outputs. The approach is empirically validated on the TOFU benchmark and demonstrates strong performance across new and existing unlearning metrics. However, the paper assumes that alternate plausible answers do not risk introducing misinformation, which is not sufficiently addressed in the evaluation. Additionally, while AltPO is shown to generalize across different forget percentages, the method is limited to QA-style factual unlearning and may not transfer well to other data modalities or task types such as dialogue or reasoning.
(Discussion) This paper aims to infer function names from stripped binaries. To better identify function names, the authors applied a technique called Domain Adaptation for fine-tuning. Their proposed model, SymGen, demonstrated good performance in various experiments, particularly showing effectiveness with unseen data. However, the paper failed to indicate that the experimental datasets used cannot be considered truly unseen, and it did not acknowledge that what is referred to as "data leakage" was actually deliberately designed.
(Discussion) DeGPT presents an innovative approach to optimizing decompiler output, demonstrating significant improvements in code readability and supporting reverse engineering analysis through a unique three-role mechanism powered by Large Language Models (LLM). However, since LLM responses are not always accurate, the implementation of the MSSC technique is crucial for preventing changes in function semantics and ensuring the correctness of optimizations. Experimental results show that DeGPT performs exceptionally well across various optimization tasks, effectively enhancing variable renaming and comment addition, thus aiding in the understanding of binary code. Nonetheless, the metrics used in the experiments may not fully capture the subjective elements of code optimization, and there may be limitations in fully explaining its real-world impact on reverse engineering analysis. Despite these limitations, the research contributes a valuable framework that significantly enhances the reverse engineering process by improving the comprehensibility and effectiveness of decompiler output.
(Discussion) The paper introduces LeakLess, a novel approach to protect sensitive data within serverless platforms from memory disclosure and transient execution attacks. LeakLess employs in-memory encryption and a separate I/O module to manage cryptographic operations, ensuring sensitive data remains encrypted within the serverless runtime's memory. This I/O module acts as a secure intermediary, handling the decryption and encryption of sensitive data during communication between serverless functions and external entities. Implemented on the Spin serverless platform, LeakLess demonstrates robust protection with minimal performance overhead. However, the clarity of the result demonstration could be improved, as there might be a possibility of plain text partially remaining in the memory region. Additionally, the reliance on tools like gcore for memory analysis might not guarantee a comprehensive search of unmapped memory regions. While LeakLess offers a significant step towards enhancing the security of serverless applications, these aspects regarding the demonstration of results and the necessity for broader evaluation across different platforms warrant further investigation. The open-source nature of LeakLess encourages community involvement and wider adoption.
(Discussion) This paper presents a novel and practical membership inference attack (MIA) against Retrieval-Augmented Generation (RAG) systems, allowing adversaries to determine if specific documents exist in the retrieval database using only model outputs. The proposed black-box and gray-box attacks are tested on medical and email datasets across multiple generative models (Flan, LLaMA, Mistral), showing high effectiveness, especially in the gray-box setting. An initial defense, involving modified RAG templates with instruction-based filtering, significantly reduces attack success for LLaMA and Mistral but is largely ineffective against Flan. While the attack is simple and requires no internal access, its reliance on prompt crafting and limited defense evaluation suggest a need for more robust, adaptive privacy-preserving techniques in future work.
(Discussion) This paper introduces S2MIA, a novel membership inference attack that targets the external databases of Retrieval-Augmented Generation (RAG) systems by exploiting semantic similarity and perplexity between a target sample and the generated output. Unlike prior attacks relying on direct prompt responses or overfitting, S2MIA infers membership by measuring how closely the system's response matches the input, even under sampling variability. Experiments across five LLMs and two datasets show that S2MIA consistently outperforms five prior MIA methods, even when standard defenses like paraphrasing, prompt modification, and re-ranking are applied. The approach is notable for its simplicity, strong performance, and model-agnostic design. However, its effectiveness weakens under paraphrasing and prompt defenses, and it relies on BLEU scores and access to model outputs and perplexity, which may not be available in all RAG systems.
(Discussion) This paper introduces Mask-Based Membership Inference Attacks (MBA), a novel approach to determine whether a target document is stored in a Retrieval-Augmented Generation (RAG) system's knowledge base. The method involves masking challenging terms in the target document, using the masked version as a query, and then assessing whether the RAG system can correctly predict the masked terms. If the masked terms are accurately predicted, it suggests the original document is likely stored in the database. The approach outperforms prior MIA methods across multiple benchmarks, achieving significantly higher ROC AUC scores. Its strengths include robustness in black-box settings, strong empirical results, and resistance to noise from internal LLM knowledge. However, it relies on high retrieval quality, handcrafted masking strategies, and tuning of multiple hyperparameters (e.g., number of masks, prediction threshold), which may limit scalability or generalizability in diverse RAG settings.
(Discussion) The Chromium browser provides browser extensions to boost user experience. As a result, browsers like Chrome distribute browser extensions via their Web Store. However, the review system in the Chrome Web Store is subject to various malicious manipulations. This paper presents FakeX, a framework to detect fake reviews by analyzing the review metadata. FakeX employs five distinct methods (e.g., temporal distribution analysis, relationship clustering, and ratio-based assessment) to identify patterns indicative of fake reviews. The authors evaluate over 1.7 million reviews and identify hundreds of suspected fake review campaigns. Furthermore, the authors uncover 86 malicious extensions ranging from data stealing to monetization. The authors highlight the potential connection between some identified fake review campaigns and malicious extensions. However, the absence of a ground truth makes the validity of the method difficult to evaluate. Furthermore, the authors do not compare their work with any existing previous work which also handles fake reviews. Nevertheless, the identification of actual malicious extensions on top of their correlation with identified fake review campaigns is an interesting finding.
(Discussion) This paper introduces VERIBIN, a system for verifying whether binary-level patches maintain functional correctness. Since security patches are often distributed as binaries without source code, traditional source-code-based verification tools are insufficient. VERIBIN employs symbolic execution to compare original and patched binaries, ensuring that modifications do not break functionality. The system improves efficiency through the Matching Path Pairs (MPP) approach and enhances accuracy with adaptive verification, allowing analyst input when necessary. The evaluation on 86 real-world patches demonstrates 93% accuracy with 0% false positives, proving its effectiveness. The paper's key strength lies in its practical applicability and its emphasis on the critical need for binary patch verification, especially in light of incidents like the XZ Utils backdoor. However, symbolic execution can be resource-intensive, leading to timeouts and memory limitations on complex binaries. Additionally, while VERIBIN can handle many structural differences, functionally equivalent but syntactically different changes remain a challenge for full automation. Despite these limitations, VERIBIN represents a significant step forward in binary patch verification and has great potential to become a powerful automated security analysis tool with further improvements.
(Discussion) This paper investigates privacy risks in Cross-App Content Sharing (Cracs), revealing how sharing activities expose user identities, social relationships, and personal data. The authors identify three key privacy issues, Sharing Behavior Tracking (SBT), where source apps infer social connections; Sharing Data Interception (SDI), where shared content is secretly collected. And Sharer Data Exposure (SDE), where a sharer’s identity is unintentionally revealed to the receiver. Using Shark, an analysis tool combining static and dynamic methods, they find that over 55% of Chinese apps and 10% of US apps exhibit privacy vulnerabilities. While the paper highlights critical concerns, it has some weaknesses. The focus on Android apps limits generalizability to iOS, and Shark’s accuracy against obfuscated code is unclear. And the discussion on legal frameworks is superficial, and there is no exploration of real-world harm from these privacy leaks. Strengthening these areas would enhance the study’s impact.
(Discussion) This paper highlights critical blind spots in SIEM rule based detection, showing that 38% of Sigma rules can be fully evaded with simple obfuscation techniques. To address this, the authors propose AMIDES, a machine-learning-based detection system that identifies evasions by comparing events to existing SIEM rules. Tested on 155 million enterprise security events, AMIDES detects 70% of evasions with zero false positives and operates 330 times faster than required for real-time deployment. A major strength of this study is its real-world validation using large-scale enterprise data and its open-source implementation, making it highly practical and reproducible. It also expands detection beyond process creation logs to include PowerShell, registry, and web requests, increasing its applicability. However, AMIDES relies on supervised learning, making it vulnerable to novel, unseen evasions, and its long-term efficiency in evolving attack landscapes needs further testing. Despite these challenges, this work demonstrates the inadequacy of rule-based SIEM detection alone and underscores the need for hybrid approaches integrating AI-driven analytics. By identifying a major cybersecurity weakness and providing a scalable solution, the study contributes significantly to the future of SIEM systems and adaptive threat detection.
(Discussion) ERASAN introduces a selective instrumentation approach to address sanitization in Rust, significantly reducing ASan’s runtime overhead without compromising its ability to detect memory bugs. By targeting raw pointer-related vulnerabilities, ERASAN eliminates redundant checks and achieves substantial performance gains. The evaluation demonstrates impressive efficiency improvements, making it a promising alternative for Rust develo However, several concerns remain. The static analysis approach may overlook certain dynamic memory issues,its optimization strategy struggles with multi-threaded environments, as memory allocation patterns in concurrent execution cannot always be statically determined. Another challenge lies in the scope of evaluation, which focuses on popular Rust crates, leaving questions about its applicability in domains with heavy foreign function interface (FFI) usage or unconventional memory management patterns. Finally, the increased compile-time overhead may hinder adoption, particularly in large-scale projects requiring frequent recompilation. While ERASAN makes notable strides in improving Rust’s memory safety tools, its scalability, multi-threading limitations, and compilation costs warrant further refinement to maximize its practicality across diverse real-world applications.
(Discussion) This paper presents PHYFU, an innovative fuzzing framework aimed at detecting logic errors in physics simulation engines (PSES) by leveraging forward and backward simulation oracles grounded in physical laws. The authors demonstrate its effectiveness across eight configurations, uncovering various errors in both forward and backward simulation processes. However, the study has several limitations. First, the reliance on manual hyperparameter tuning for oracle thresholds and perturbation magnitudes makes the approach labor-intensive and less reproducible. Second, while PHYFU claims general applicability, its evaluation is confined to a limited set of scenarios and engines, leaving its effectiveness in diverse or unconventional physical simulations uncertain. Third, the framework's detection of "unapparent errors," which lack observable patterns, raises concerns about false positives and the robustness of its oracles. Finally, although PHYFU highlights errors caused by simulation inaccuracies, it does not explore their downstream impact on applications such as robotics or training agents, missing an opportunity to contextualize the significance of the findings. Despite its novelty, these constraints highlight challenges in scalability and broader applicability.
(Discussion) The study explores instruction-based backdoor attacks on large language models (LLMs), targeting their instruction-following capabilities through crafted prompts embedding backdoor instructions. It categorizes attacks into word-level, syntax-level, and semantic-level, showing high attack success rates (ASR) while maintaining clean input utility. Larger models like GPT-4 and Claude-3 exhibit higher ASRs, making them more susceptible than smaller models such as LLaMA2 and Mistral. However, the attack's effectiveness depends heavily on trigger design, including length, position, and complexity, with semantic-level attacks being stealthier but less versatile in simpler tasks. The study highlights challenges in defending against such attacks, as existing detection methods like intent analysis yield high false alarm rates and limited accuracy in identifying more covert triggers. Practical limitations include the reliance on static prompt structures, assuming attackers have precise knowledge of prompt formats and tasks, which may not generalize to dynamic or adversarial real-world scenarios. Additionally, the approach has not been tested extensively across diverse LLM architectures or specialized domains, leaving gaps in understanding broader vulnerabilities and mitigation strategies.
(Discussion) The paper introduces PLeak, an innovative framework designed to extract confidential system prompts from LLM applications, which are often protected as intellectual property. PLeak automates the process using a closed-box optimization approach, relying on shadow LLMs and datasets to simulate the target environment. It uses techniques like incremental query optimization and response aggregation to reconstruct system prompts effectively. The framework significantly outperforms previous methods, achieving high accuracy in metrics such as Exact Match and Semantic Similarity, and successfully reconstructs prompts in 68% of real-world tests on the Poe platform. However, PLeak has limitations, including its dependence on shadow environments that closely resemble the target, potential scalability issues with computationally intensive optimizations, and only partial exploration of defenses like output filtering or obfuscation. While it demonstrates critical security risks in LLM applications, the lack of broad evaluation across diverse platforms and limited engagement from reported parties like Poe raise concerns about its practical implications and ethical considerations.
2024
(Discussion) This paper introduces D-CAPTCHA, an innovative active defense mechanism against real-time deepfakes. Unlike traditional passive detection methods that rely on analyzing patterns or artifacts, D-CAPTCHA challenges the capabilities of deepfake models by requiring the attacker to perform tasks that are easy for humans but difficult for AI to replicate, such as humming a tune or speaking with emotion. Experimental results demonstrate that D-CAPTCHA significantly enhances detection accuracy, achieving 91-100% compared to state-of-the-art methods. While it offers long-term effectiveness and extensibility in combating audio-based deepfake threats, its practical application is limited to real-time scenarios, and it may impact user experience if not carefully implemented. Furthermore, it is worth noting a minor inconsistency in Table 1, where the acronym "R" is used for both Repeat Accent and Raspberry, which could be corrected for clarity. Overall, D-CAPTCHA represents a groundbreaking approach to addressing the growing threat of deepfake technology and highlights the potential for proactive AI-driven security solutions that adapt to evolving adversarial capabilities.
(Discussion) Since the advent of Large Language Models (LLMs), there has been increasing interest in the concept of 'Jailbreak Prompts' for these LLMs. Since LLMs are trained on a large corpus of data, they can generate harmful content. As a result, LLM services provide 'Aligned Language Models' that aim to prevent generating harmful content. The 'Jailbreak Prompts' are prompts that allow an adversary to bypass the defense and cause LLMs to generate harmful content. This paper conducts a measurement study on existing Jailbreak Prompt methods to answer three research questions: 'What are jailbreak strategies and how do they work?', 'How do humans develop and execute jailbreak attacks?', and 'How to automate the process of jailbreaking?'. While the paper overviews the current landscape of jailbreak prompts, the metrics defined in the paper (Expected Maximum Harmfulness (EMH) and Jailbreak Success Rate(JSR)) are difficult to comprehend. For instance, in Table 3 the authors present the metrics for pairs of jailbreak prompts and malicious queries. Unfortunately, it is difficult to comprehend the result as EMH does not seem to be normalized (exceeding 1.0 in certain cases) and the overall JSR is very low, causing the reader to question the effectiveness of the attack.
(Discussion) This paper delve into the innovative contributions of RustSan, assessing its implications in the Rust programming ecosystem. While the integration of AddressSanitizer into Rust programming through selective instrumentation significantly reduces runtime overhead, it also raises questions about the balance between optimization and comprehensiveness of memory safety checks. One needs to consider the scenarios where safe and overlapping memory regions are crucial yet may be overridden inadvertently in corner cases. And all hopes on static analysis for detection of unsafe blocks may fall short of considering dynamic interactions which only appear at runtime and thus present challenges in practical applications. While this may sound promising, performance gains shown in benchmarks will have to be investigated further in larger-scale applications with less predictable memory access patterns. Finally, though the approach taken by RustSan mitigates some of the inherent trade-offs between flexibility and safety in Rust, it remains important to establish the robustness of the tool in the face of complex programming paradigms and its ease of integration into existing workflows for developers. This prompts an ongoing evaluation of its long-term reliability and adaptability.
(Discussion) This paper makes a valuable contribution by systematically revealing undocumented APIs and their potential security exploits in popular “super apps” such as WeChat and TikTok. By leveraging both static and dynamic analysis, the authors’ tool, APIScope, successfully identifies hidden APIs and demonstrates their risks, thereby advancing our understanding of super app ecosystems and offering actionable guidance for platform vendors and developers. However, several limitations remain. The study focuses exclusively on Android versions of super apps using the V8 engine, narrowing the generalizability of its findings and excluding large market segments, like iOS platforms or super apps employing different JavaScript engines. Moreover, while APIScope identifies hidden APIs and validates certain vulnerabilities through test miniapps, it does not fully explore how organizational factors—such as development practices, code review processes, or internal security policies—contribute to these undocumented APIs. The paper also falls short of presenting robust, long-term defenses or systematic industry-level guidelines for mitigating undocumented APIs. A deeper integration of automated patching, secure coding frameworks, and clearer vendor-side standardization efforts would strengthen the practical impact. Future work should strive to broaden platform scope, thoroughly investigate root causes behind the emergence of undocumented APIs, and propose more comprehensive preventative strategies.
(Discussion) The paper introduces neural phishing, a method to extract sensitive information from large language models (LLMs) by injecting benign-looking poisoned data into their training datasets. The attack has three phases: poisoning during pretraining, memorization during fine-tuning, and information extraction during inference. Success rates range from 10% to 80%, depending on the model size, data duplication, and attacker knowledge. Larger and overtrained models are more vulnerable. Standard defenses like deduplication and differential privacy are ineffective. The work highlights significant privacy risks in LLMs and emphasizes the need for advanced defenses. While the study is significant, its limitations include reliance on poisoned data appearing before secrets in training, difficulty in fully preventing memorization of sensitive patterns, and the low likelihood of individuals leaking sensitive data directly to LLMs.
(Discussion) The paper presents a novel method to infer whether a document was included in the training dataset of large language models (LLMs). Using a black-box approach based on predicted token-level probabilities and normalization techniques, the authors develop a meta-classifier to distinguish between member and non-member documents. Evaluated on OpenLLaMA with datasets from Project Gutenberg and ArXiv, the method achieves AUC scores of 0.856 for books and 0.678 for papers, demonstrating its effectiveness. The study highlights that even small LLMs retain significant memorization, raising concerns about privacy and transparency in LLM training data. However, the approach relies on known datasets, limiting its applicability to proprietary models without accessible training data. It also assumes that non-member documents are similar but temporally distinct, which may not always hold. Additionally, the method's focus on black-box inference may not address vulnerabilities in fine-tuned or modified LLMs, warranting further research into privacy mitigation strategies.
(Discussion) This study effectively highlights a critical security vulnerability in containerized applications, revealing that 8.5% of analyzed Docker images contain sensitive information such as cryptographic keys or API secrets. The large-scale dataset and rigorous filtering enhance the reliability of findings, providing strong evidence that secret leakage is a systemic issue rather than isolated errors. This insight underscores the need for structural changes in the Docker paradigm, offering valuable recommendations such as improved secret scanning tools and stricter default configurations. However, the reliance on regex-based detection may limit the identification of unconventional leaks, and the inability to validate API secrets due to ethical constraints reduces the completeness of the analysis. Furthermore, the discussion could have delved deeper into human and organizational factors, such as user training, to address the root causes of these leaks. Despite these limitations, the study makes significant contributions by exposing the widespread impact of secret leakage, affecting over 275,000 Internet-facing hosts, and providing a foundation for collaborative efforts to mitigate these vulnerabilities. This research holds substantial value for improving container security practices and guiding future innovations in secret management.
(Discussion) After the advent of deep learning (DL) models, several practical attacks against DL have surfaced. Consequently, the research community developed a complementary vetting technique to detect such attacks. However, these techniques do not address how to retrieve these DL models for inspection. Existing state-of-the-art memory forensic techniques rely on static data structure recovery, which cannot recover complex data structures used in DL software stacks. Furthermore, ML frameworks employ advanced garbage collection and memory management optimization, which obfuscates in-memory objects. Moreover, the layer weights are stored in a consecutive GPU memory while the context of these objects is stored in the CPU data structure. Lastly, white-box analysis techniques must rehost the DL model in a live environment for inspection. This paper proposes AiP, an automated system for recovering DL models from meory and rehosting them into a live process. AiP requires no prior knoweldge of the particular model. In the evaluation using LISA, CIFAR-10, and IMDB dataset, AiP successfully recovered 30 different models and achieved 100% recover accuracy. The paper demonstrates its practicality by recovering popular Python-based frameworks (Pytorch and Tensorflow), however acknowledges the existance of other ML frameworks (e.g., HuggingFace).
(Discussion) This paper presents BUDAlloc, a novel one-time allocator (OTA) designed to address use-after-free (UAF) vulnerabilities by decoupling virtual address management from the kernel. Unlike conventional approaches, BUDAlloc shifts virtual memory operations to the user level, reducing system call overhead and improving performance, especially in multi-threaded applications. Using eBPF-based page fault handling, BUDAlloc efficiently manages virtual aliases and batches memory operations to minimize fragmentation. It offers two modes—detection and prevention—allowing users to prioritize either bug detection or performance. Evaluations show that BUDAlloc improves performance by 15% over DangZero and reduces memory overhead by 61% compared to FFmalloc, making it a scalable and practical solution for UAF bug detection without modifying existing binaries.
(Discussion) The paper, Exploring ChatGPT’s Capabilities on Vulnerability Management, examines ChatGPT's ability to assist in six stages of vulnerability management, utilizing a dataset of 70,346 samples. The authors compare ChatGPT's performance with state-of-the-art (SOTA) methods, assessing tasks like bug report summarization and vulnerability repair. While ChatGPT demonstrates promising potential in some areas, such as summarizing bug reports, challenges persist, particularly in patch correctness assessment. The paper's strengths include a comprehensive dataset and innovative exploration of prompt engineering techniques. However, it faces limitations in generalizing ChatGPT’s performance across all tasks, pointing to future research needs for refining its application in security tasks. This study is significant in highlighting ChatGPT’s growing role in automated vulnerability management, with the potential for advancing research in the domain.
(Discussion) This paper presents WEBRR (Web-based Enterprise Record and Replay), a novel forensic system designed for replaying and investigating web-based attacks in modern web environments. The authors address the limitations of existing forensic analysis tools in dealing with the complexity of modern web applications and browsers. WEBRR achieves deterministic replay of web attacks by leveraging the single-threaded nature of JavaScript and implementing a comprehensive recording and replay mechanism for various web components, including the Document Object Model (DOM), network requests, and asynchronous operations. The system is implemented as an extension to the Chromium browser and demonstrates high accuracy in replaying both benign websites and malicious attacks across multiple platforms (Linux, Windows, and Android). The authors evaluate WEBRR against existing solutions like WebCapsule, Mugshot, and RR, showing superior performance in replaying complex web attacks. The system exhibits low runtime overhead (median 2.94% increase in page load time) and reasonable storage requirements (17 MB per minute of browsing). While WEBRR has some limitations, such as incomplete support for certain web technologies (e.g., WebRTC, IndexedDB), the authors argue that these can be addressed with additional engineering effort. Overall, WEBRR represents a significant advancement in web forensics, enabling more accurate and interactive investigations of web-based attacks.
(Discussion) This paper presents a novel deepfake attack called Foice. This attack generates a synthetic voice of a victim using just a single face image, without requiring any voice samples. The generated voice is realistic enough to fool commercial voice authentication systems, such as those used by WeChat, Microsoft Azure, and other platforms. The core idea behind Foice is to learn the partial correlation between face and voice features, and then combine these with random face-independent voice features generated from a Gaussian distribution. The attack's effectiveness was demonstrated through real-world experiments on several authentication systems and voice assistants, where it showed high success rates, indicating significant vulnerabilities in these systems. The study emphasizes the importance of new defenses against such attacks, as current systems lack sufficient protection against deepfake threats based solely on facial images.
(Discussion) The paper provides a comprehensive analysis of the MITRE ATT&CK evaluations, aiming to address the limitations of the raw evaluation results. The paper's strengths lie in its introduction of new analysis methods, including whole-graph analysis and holistic assessments, to gain deeper insights into EDR system capabilities. The authors' reconstruction of attack scenarios and subsequent analysis shed light on EDR systems' attack reconstruction, behavior correlation, and overall detection performance. However, the paper could be improved by addressing the lack of access to crucial information like false positive alarm volume, response time, and raw data, which limits the scope of the analysis. Despite this limitation, the paper offers valuable contributions to the field by providing a systematic interpretation of MITRE ATT&CK evaluation results, aiding researchers, practitioners, and vendors in understanding and improving EDR systems.
(Discussion) Code coverage faces a major challenge in that it only reflects a small part of a program's structure, leaving some crucial program constructs uncovered. To address this issue, this work proposes data coverage for guided fuzzing - a technique that focuses on detecting novel constant data references and maximizing their coverage. Additionally, real-world fuzzing practices were optimized by classifying data access according to semantics and designing customized collection strategies. This is crucial because improper handling of constant data can significantly impact fuzzing throughput. To further improve fuzzing efficiency, novel storage and utilization techniques were developed. Finally, libFuzzer was enhanced with data coverage capabilities and submitted to Google's FuzzBench for evaluation. In this evaluation, the proposed approach outperformed many state-of-the-art fuzzers and achieved the best coverage score in the experiment. As a result, using this enhanced code coverage, 28 previously unknown bugs in OSS-Fuzz projects were discovered.
(Discussion) This paper presents RACING, a novel approach for efficient root cause analysis (RCA) in fuzzing. By leveraging counterexamples and reinforcement learning, RACING intelligently guides the fuzzing process to quickly identify the root cause of crashes. The experimental results demonstrate that RACING significantly outperforms the state-of-the-art technique, Aurora, in terms of both speed and accuracy. This work holds significant implications for improving the efficiency and effectiveness of vulnerability detection and analysis in software. However, limitations exist, such as the reliance on source code and the lack of support for compound predicates, which offer avenues for future research. Overall, RACING represents a promising advancement in automated RCA for fuzzing, with the potential to significantly impact the field of software security.
(Discussion) This paper explores the memory access patterns of language models (LMs), particularly focusing on the challenges they face with random access to memorized information. The authors demonstrate that while LMs can sequentially reproduce stored content effectively, they struggle with accessing information in random segments, especially in the middle of memorized sequences. To address this, the paper proposes two strategies that "recitation" (having the model first recall the content) and "permutation" (shuffling sentence order) and shows through experiments that these methods can significantly improve the model's performance. The study provides valuable insights into the limitations of memory access in LMs and suggests practical ways to enhance their performance in real-world applications. However, the research is limited to decoder-only models and does not explore larger models beyond 7 billion parameters, which could offer further insights. Additionally, experiments are conducted on a fixed-size text corpus, leaving questions about scalability to larger pretraining datasets. Despite these limitations, the paper makes an important contribution to understanding and improving memory access in language models.
(Discussion) LLMs demonstrate remarkable capabilities in various complex tasks. Recent research has delved into the possibility of enhancing the performance of these LLMs by incorporating human-annotated data. Unfortunately, this method is limited in scalability and underperforms in specific scenarios. This paper proposes a technique that can automatically generate natural language rationales using the output attribution scores that capture the influence of each feature on model predictions. The new framework, AMPLIFY, improves the prediction accuracy to about 10-25%, including those where prior approaches relying on human-annotated rationales fall short. A fundamental limitation of AMPLIFY is that it inherits the limitations of LLMs and Post hoc explanation methods.
(Discussion) The authors of the paper present a clever and straightforward approach to enhance the safety of large language models (LLMs). The idea is to have the LLM essentially double-check its own work for any potentially harmful content. When the LLM receives a prompt that could lead to a harmful response, it generates the text as usual. But then, this response is passed to another LLM that's been instructed to act as a filter. This filter LLM reads the generated text and makes a judgment call by evaluating "harmfulness". The beauty of this method lies in its simplicity - doesn't require any retraining or tweaking of the original LLM, also it doesn't involve any complex pre-processing of the input. They tested this "LLM self-defense" mechanism on two popular models, GPT-3.5 and LLaMA 2. In many cases, they were able to virtually eliminate the generation of harmful content.
(Discussion) In this paper, a new weakness in Large Language Models (LLMs) is exposed that doesn’t rely on creating special prompts, known as jail-breaking. Instead, this method, called model interrogation, uses the fact that even when an LLM refuses to answer a harmful question, the damaging reply might still be hidden in the less obvious parts of its output. By accessing the list of probable responses (top-k token predictions) available in many open-source and commercial LLMs, a person with bad intentions can manipulate the model to reveal these harmful answers. They do this by choosing less likely words from the list at certain points in the response. This technique proves to be not only different but also more effective than traditional jail-breaking, with a 92% success rate compared to 62%, and it works 10 to 20 times faster. The harmful content brought out by this method is also of higher quality. Moreover, combining model interrogation with jail-breaking methods greatly improves results over using either technique alone. The study also shows that LLMs made for specific tasks like programming can still be tricked into giving harmful responses using this method.
(Discussion) This paper introduces LLM Compiler, a novel large language model designed for code optimization and compiler tasks. Building upon the Code Llama model, LLM Compiler is specifically trained to understand intermediate representations (IR) and assembly code, showing improved performance in code generation and compiler emulation tasks. The model demonstrates strong capabilities in handling compiler-related tasks and outperforms previous models in various benchmarks. However, the paper notes some limitations. The primary issue is the model's fixed input sequence length of 16k tokens, which restricts its ability to handle very large codebases effectively. Despite efforts to split large translation units, some remain too large for the model to process. Additionally, the accuracy of the model's outputs requires rigorous evaluation and verification to ensure correctness, as any suggested optimizations should be thoroughly tested. Despite these limitations, the paper makes significant contributions to the field of compiler optimization and provides a solid foundation for future research.
(Discussion) The paper 'Unlearning Bias in Language Models by Partitioning Gradients' presents a novel technique, partitioned contrastive gradient unlearning (PCGU), to debias pretrained masked language models. PCGU selectively optimizes weights that contribute to biases by calculating gradients for contrastive sentence pairs. Evaluations using StereoSet and CrowS Pairs datasets demonstrate PCGU's effectiveness in reducing gender-profession bias with minimal impact on language modeling performance. Additionally, PCGU shows potential in mitigating biases across other domains like race and religion. However, it would have been better if there were additional experiments depending on the size of the language model. Also This technique highlights the need for ongoing research to develop robust strategies for addressing bias in language models.
(Discussion) ProPILE, a novel probing tool designed to assess the risk of Personally Identifiable Information (PII) leakage in Large Language Models (LLMs), demonstrates the feasibility of extracting PII through strategic prompt engineering. By utilizing black-box probing for data subjects and white-box probing for LLM service providers, ProPILE enables the assessment of PII leakage in models like OPT-1.3B trained on the Pile dataset. Experiments reveal that the likelihood of target PII generation increases with more specific prompts or additional linked PII details, while white-box probing with access to a limited subset of training data significantly amplifies this leakage potential. These findings underscore the effectiveness of ProPILE as an assessment tool and highlight the need for ongoing research and robust mitigation strategies to address privacy vulnerabilities in LLMs.
(Discussion) This paper proposes a bi-directional transformer-based guessing framework, referred to as PassBERT, which applies the pre-training/fine-tuning paradigm to password guessing attacks. First, a model pre-trained on the general password distribution was prepared, which was then fine-tuned on three specifically designed attack approaches. These methods reflect real-world attack scenarios and include: 1) conditional password guessing, which recovers the complete password given a partial one; 2) targeted password guessing, which compromises the password of a specific user using personal information; and 3) adaptive rule-based password guessing, which selects rules to generate rule-transformed password candidates. The experimental results show that the fine-tuned models can outperform state-of-the-art models by 14.53%, 21.82%, and 4.86% in the three attacks, respectively, demonstrating the effectiveness of bi-directional transformers on downstream guessing attacks. Furthermore, a hybrid password strength meter was proposed to mitigate the risks from these three types of attacks.
(Discussion) This paper introduces an innovative multi-stage ranking system for extracting and annotating adversarial techniques from threat intelligence reports, leveraging language models fine-tuned for cybersecurity tasks. The system demonstrates significant improvements in accuracy and recall compared to previous methods, with enhanced performance on verbose datasets like MITRE ATT&CK Reports and WeLiveSecurity. The comprehensive approach, including testing across various datasets and the introduction of a public dataset with 6.6K annotations, strengthens the study's contributions to the field. However, the observed performance disparity across datasets highlights the need for further refinement to handle concise descriptions more effectively. Overall, the study advances automated threat technique annotation and provides a solid foundation for future cybersecurity research and development.
(Discussion) Detecting anomalies within system logs is paramount to protecting the system from an attack or malfunction. Traditional methods employ regular expressions or machine learning models to identify anomalous events. These approaches depend on handcrafted features and are unable to capture temporal information. For example, malicious logs could be benign on their own, but the collection of these logs could be malicious. Recently, deep learning models such as recurrent neural networks (RNNs) have been widely used to evaluate these sequences and can capture temporal information. However, RNNs cannot encode contextual information in a bi-directional manner. This paper proposes a log anomaly detection model based on BERT. BERT can capture contextual information in a bi-directional manner making it suitable for this task. The model is evaluated on three datasets: Hadoop Distributed File System (HDFS), BlueGene/L Supercomputer System (BGL), and Thunderbird-mini dataset. The baseline models are Principal Component Analysis (PCA), One-Class SVM (OCSVM), IsolationForest (iForest), LogCluster, DeepLog, and LogAnomaly. Their proposed model demonstrates superior performance over all baseline state-of-the-art models in all datasets. However, the detailed process in training is not described. For example, BERT splits the training step into a pretraining and finetuning phase. However, LogBERT seems to only discuss the pretraining phase and have no mention about the fine tuning phase. As a result, it is not clear if the fine tuning phase is excluded or it is not mentioned.
(Discussion) This paper proposes an attack method that causes aligned language models to generate undesirable behaviors. Aligned language models refuse to respond to harmful queries. To enable these language models to respond to harmful queries, this paper attaches a suffix to the query. This approach makes the LLM generate a positive response instead of refusing to answer. Positive responses to harmful queries were obtained from ChatGPT, Bard, Claude, LLaMA-2-Chat, Pythia, and Falcon, with a much higher success rate for GPT-based models. The evaluation section only demonstrates the high success rate of the proposed attack method. It would have been better if the paper included the limitations of this study and a model ablation study section.
(Discussion) The paper titled 'Membership Inference via Backdooring' presents a novel approach to addressing data privacy concerns in machine learning by introducing a method called Membership Inference via Backdooring (MIB). The main contribution of MIB lies in its ability to mark a small number of data samples, which, when used by an unauthorized party to train a model, allows the data owner to later identify this misuse through black-box queries and statistical hypothesis testing. However, several limitations and challenges accompany this promising approach. Firstly, the success of MIB hinges on the ability to inject backdoor triggers in a manner that remains undetectable by the unauthorized party. While the paper discusses techniques to make the triggers imperceptible, there is an inherent risk that sophisticated adversaries could develop detection methods to identify and neutralize these backdoors. Furthermore, the approach's reliance on statistical hypothesis testing to provide guarantees for inference results, while innovative, may still be vulnerable to model-specific variations and adversarial defenses. However, further exploration is needed to assess the robustness of MIB across various types of datasets and models not covered in the experiments of the paper. Additionally, it is regrettable that the examples from the various datasets experimented in Figure 5 were not also included.
(Discussion) The paper titled 'Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations' introduces Llama Guard, a model developed to enhance safety in human-AI interactions. The model leverages the Llama2-7b architecture, fine-tuned to classify and mitigate safety risks in both user prompts and AI responses. The core contribution is a comprehensive safety risk taxonomy that guides the classification process, encompassing categories like violence, hate speech, sexual content, and self-harm. Llama Guard outperforms existing moderation tools on benchmarks such as the OpenAI Moderation Evaluation dataset and ToxicChat. The model supports customizable and adaptive use through zero-shot and few-shot learning, and its weights are publicly available for further research and development. However, the work has some limitations. Llama Guard's common sense knowledge is constrained by its training data, which can lead to incorrect judgments. Its performance in non-English languages is not guaranteed, and the quality of fine-tuning labels may not comprehensively cover all policy aspects, leading to subpar performance in some cases.
(Discussion) This study collects and analyzes data from human participants through malware classification games, clearly revealing the differences between human and machine learning models. This is an original approach not seen in previous studies. However, the number of samples used in the experiment is limited to 20, which may be somewhat insufficient to represent the overall malware classification problem. Nevertheless, their results provide practical insights that can be applied directly to training malware analysis experts and improving ML models. For example, they propose integrating dynamic behavior analysis of human experts into ML models. This work provides important insights into how human experts and ML models classify malware, and suggests that a hybrid approach that incorporates human intuition and experience into ML models could be highly effective in future malware defense.
(Discussion) Let there be a video of a vehicle moving backward. An anomaly detection by frame alone would fail to capture any anomalies; instead, the model requires a sequence of frames to identify the anomaly. This paper proposes a Transformer-based anomaly detection in aerial videos from an Unmanned Aerial Vehicles (UAVs). By leveraging the Transformer encoder, they claim that the model can preserve spatiotemporal information. The main contribution comprises an annotated dataset for realistic anomalous events, a benchmark for anomaly detection, and a baseline model ANDT. ANDT exhibits the best performance for certain scenes but does not do so for all scenes tested in the paper. Initial claims suggest that previous models do not preserve spatiotemporal information. However, previous models demonstrate decent performance on the provided dataset. In Figure 4, the MemAE model can infer the vehicle moving backward as an anomalous event, demonstrating its ability to preserve spatiotemporal information. Furthermore, the train test split in Table 1 is odd yet there is no mention why it was done in this way. For example, the test set is larger than the train set in the Bike roundabout scene.
(Discussion) In many ways, the compilation process is destructive. High-level constructs, variable names, and comments present in the source code are often lost. Nevertheless, decompilers aim to retrieve the source from a binary. Decompilation results vary between decompilers. So, what is a good decompilation? One could argue that fewer goto instructions indicate good decompilation. Because multiple gotos signify the decompiler failed to structure the control flow. This paper argues that good decompilation is similar to the source code and points out that 3,754 goto instructions are present in the Linux kernel. Since a goto instruction could be intentional by the developer, they argue we should separate intended gotos from unintended gotos. The authors investigate the gcc compiler to identify the transformation passes that cause unintended gotos. Afterward, they propose a structuring algorithm that deoptimizes code to remove unintended gotos while preserving intended ones. SAILR, the proposed structuring algorithm, is implemented on the angr decompiler. SAILR demonstrates decent performence against state-of-the-art decompilers. The paper evaluates SAILR with 7,355 functions from 26 popular Debian packages. However, the number of functions is oddly low, considering the number of packages used.
(Discussion) This paper proposes the Membership Inference method 'MIN-K% PROB' in LLM(Large Language Model). The problem presented in this paper is the challenge of detecting pretraining data. This is because LLM developers do not open the data used for pertaining, and since pretraining involves training on data instances once at a time, it makes detection even more difficult. Hence, this paper assumes that the Membership Inference Detector cannot know the distribution of pretraining data. This means that there is no Reference Model (e.g., Shadow Model) used in Membership Inference techniques. Consequently, this paper proposes the Reference-free Method, MIN-K% PROB. The 'Min-K% PROB' method selects outlier tokens with low token probability to create a set, and then uses the average of this set as the threshold for inferring between members and non-members. This method seems to be efficient as it infers membership based on the token-level probability during the pretraining. Additionally, using this method as an evaluation for Unlearning techniques, as demonstrated in the case study, would be beneficial.
(Discussion) The paper investigates the application of prompt learning with Large Language Models (LLMs) such as GPT-3 and T5 to address the issue of toxic online content. It focuses on three primary tasks: toxicity classification, toxic span detection, and detoxification, showing that prompt learning can perform as well as or even better than traditional models specifically trained for these tasks. Notably, this approach achieves a 10% improvement in toxicity classification and significantly lowers toxicity scores in detoxification tasks while maintaining the semantic integrity of the content. The study is distinguished by its innovative use of prompt tuning for managing toxic content and its thorough evaluation across five model architectures and eight different datasets, which verifies the method's effectiveness and efficiency. However, despite these strengths, the approach's dependency on the quality of datasets and its ability to generalize to unseen toxic content remain potential weaknesses. Furthermore, the complexity involved in designing effective prompts and the possible misuse of the techniques are also concerns.
(Discussion) This paper proposes 'ANUBIS', a provenance graph-based supervised APT detection framework. ANUBIS is a method leveraging a BNN(Bayesian Neural Network) to overcome limitations in current APT detection methods using provenance graphs. The authors hypothesize that an attacker cannot breach the probability graph used to train ANUBIS, nor does it violate the event logging mechanism of the host system. These assumptions provide a strong premise for security scenarios, but they have limitations that do not take into account the practical considerations. However, this paper is significant because it has demonstrated that it is effective in predicting the nature of the activity by introducing a new graph neighbor encoding method.
(Discussion) This paper is a measurement study on the IoT Malware Lifecycle. It collects around 166K Linux-based IoT malware samples collected over a year to measure the characteristics of IoT Malware. The paper concludes with some characteristics that differentiate IoT malware from traditional malware, such as most IoT malware being a variant of the Mirai botnet. However, there does not seem to be unexpected findings. A slight complaint is that figures within the paper had unclear captions. Although the paper explains the concepts, some figures are difficult to understand.
(Discussion) In this paper, a dataset composition for testing the latest BCSD (Binary Code Similarity Detection) techniques is proposed, and analysis of test results for various BCSD models, including the latest GNN-based BCSD and Embedding-based BCSD, has been performed. However, in the evaluation section, it is necessary to divide the results for similarity and similarity lacking into separate tables to enhance clarity. Additionally, there is a lack of explanation for the many abbreviations used in the dataset composition, making it difficult to understand.
(Discussion) The paper discussion focuses on an innovative approach to optimizing the reward function of a Reinforcement Learning (RL) model to mitigate undesired behaviors. The authors detail a three-stage algorithm comprising exploration, quantization, and learning, each critical to the model's development. Notably, the paper demonstrates the model's effectiveness through three distinct evaluations, addressing the reduction of toxicity, improvement over negative baselines, and minimization of repetitive actions. This structured evaluation underscores the model's capability to unlearn specific unwanted behaviors, which outperforms both baselines and state-of-the-art RL methods.
In this paper, the focus is on defining three undesirable behaviors and proposing three methods (e.g. reward function) to forget each behavior. It seems inefficient to train a model separately for each behavior.
(Discussion) This paper designs a multidimensional detection framework to detect lateral movement behavior against APT attacks in an intranet environment based on the SMB protocol. However, contrary to this purpose, the experimental results are unfortunate in that the purpose and results differ because they are about how well malware has been classified.
(Discussion) Systems generate logs to record internal events. These logs are used after a cyber incident for forensic investigation. Unfortunately, the system generates numerous logs during its runtime and majority of the analysis is manual. This paper proposes a deep autoencoder for anomaly detection to assist analyzers by highlighting anomalous events in the Linux kernal system logs.
Anomaly detection is a common application for autoencoders. It is utilized to analyze network traffic, logs, etc. It is difficult to identify what is different from previous work and this paper. Furthermore, the proposed model is evaluated with old dataset against ML methods such as SVM. It would have been preferable to see the model’s performance on up-to-date dataset against state-of-the-art models.
(Discussion) There are efficient ways to represent binary functions using a tokenizer (e.g., BPE) in assembly language. I'm curious about why the suggestion is to divide them into seven features like Vex-IR instructions, LibcCalls, Constants, etc., and create specific tokenizers for each feature to generate one-hot vectors. I also wonder if this approach is genuinely effective. The explanation about whether this method of tokenization has a positive impact on BCSD (Binary Code Similarity Detection) performance seems insufficient.
(Discussion) The paper utilizes adversarial examples to retain the original decision boundary during unlearning. This approach is intriguing yet it raises questions as to what it means to “forget” a data point. The paper states that misclassification signifies forgetting yet “Right to be Forgotten” seems to imply the removal of a certain training data.
Malware datasets inevitably contain incorrect labels due to the shortage of expertise and experience needed for sample labeling. Previous research demonstrated that a training dataset with incorrectly labeled samples would result in inaccurate model learning. To address this problem, researchers have proposed various noise learning methods to offset the impact of incorrectly labeled samples, and in image recognition and text mining applications, these methods demonstrated great success. In this work, we apply both representative and state-of-the-art noise learning methods to real-world malware classification tasks. We surprisingly observe that none of the existing methods could minimize incorrect labels’ impact. Through a carefully designed experiment, we discover that the inefficacy mainly results from extreme data imbalance and the high percentage of incorrectly labeled data samples. As such, we further propose a new noise learning method and name it after MORSE. Unlike existing methods, MORSE customizes and extends a state-of-the-art semi-supervised learning technique. It takes possibly incorrectly labeled data as unlabeled data and thus avoids their potential negative impact on model learning. In MORSE, we also integrate a sample re-weighting method that balances the training data usage in the model learning and thus handles the data imbalance challenge. We evaluate MORSE on both our synthesized and real-world datasets. We show that MORSE could significantly outperform existing noise learning methods and minimize the impact of incorrectly labeled data.
(Abstract) Pre-trained models for Natural Languages (NL) like BERT and GPT have been recently shown to transfer well to Programming Languages (PL) and largely benefit a broad set of code-related tasks. Despite their success, most current methods either rely on an encoder-only (or decoder-only) pre-training that is suboptimal for generation (resp. understanding) tasks or process the code snippet in the same way as NL, neglecting the special characteristics of PL such as token types. We present CodeT5, a unified pre-trained encoder-decoder Transformer model that better leverages the code semantics conveyed from the developer-assigned identifiers. Our model employs a unified framework to seamlessly support both code understanding and generation tasks and allows for multi-task learning. Besides, we propose a novel identifier-aware pre-training task that enables the model to distinguish which code tokens are identifiers and to recover them when they are masked. Furthermore, we propose to exploit the user-written code comments with a bimodal dual generation task for better NL-PL alignment. Comprehensive experiments show that CodeT5 significantly outperforms prior methods on understanding tasks such as code defect detection and clone detection, and generation tasks across various directions including PL-NL, NL-PL, and PL-PL. Further analysis reveals that our model can better capture semantic information from code. Our code and pre-trained models are released at https: //github.com/salesforce/CodeT5.
(Abstract) Code is seldom written in a single left-to-right pass and is instead repeatedly edited and refined. We introduce INCODER, a unified generative model that can perform program synthesis (via left-to-right generation) as well as editing (via masking and infilling). InCoder is trained to generate code files from a large corpus of permissively licensed code, where regions of code have been randomly masked and moved to the end of each file, allowing code infilling with bidirectional context. Our model is the first large generative code model that is able to infill arbitrary regions of code, which we evaluate in a zero-shot setting on challenging tasks such as type inference, comment generation, and variable re-naming. We find that the ability to condition on bidirectional context substantially improves performance on these tasks, while still performing comparably on standard program synthesis benchmarks in comparison to left-to-right only models pretrained at similar scale. Our models and code are publicly released.
(Abstract) Program synthesis or code generation aims to generate a program that satisfies a problem specification. Recent approaches using large-scale pretrained language models (LMs) have shown promising results, yet they have some critical limitations. In particular, they often follow a standard supervised fine-tuning procedure to train a code generation model only from the pairs of natural-language problem descriptions and ground-truth programs. Such paradigm largely ignores some important but potentially useful signals in the problem specification such as unit tests, which thus often results in poor performance when solving complex unseen coding tasks. To address the limitations, we propose “CodeRL”, a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning (RL). Specifically, during training, we treat the code-generating LM as an actor network, and introduce a critic network that is trained to predict the functional correctness of generated programs and provide dense feedback signals to the actor. During inference, we introduce a new generation procedure with a critical sampling strategy that allows a model to automatically regenerate programs based on feedback from example unit tests and critic scores. For the model backbones, we extended the encoder-decoder architecture of CodeT5 with enhanced learning objectives, larger model sizes, and better pretraining data. Our method not only achieves new SOTA results on the challenging APPS benchmark, but also shows strong zero-shot transfer capability with new SOTA results on the simpler MBPP benchmark.
(Abstract) Modern disassembly tools often rely on empirical evaluations to validate their performance and discover their limitations, thus promoting long-term evolvement. To support the empirical evaluation, a foundation is the right approach to collect the ground truth knowledge. However, there has been no unanimous agreement on the approach we should use. Most users pick an approach based on their experience or will, regardless of the properties that the approach presents. In this paper, we perform a study on the approaches to building the ground truth for binary disassembly, aiming to shed light on the right way for the future. We first provide a taxonomy of the approaches used by past research, which unveils five major mechanisms behind those approaches. Following the taxonomy, we summarize the properties of the five mechanisms from two perspectives: (i) the coverage and precision of the ground truth produced by the mechanisms and (ii) the applicable scope of the mechanisms (e.g., what disassembly tasks and what types of binaries are supported). The summarization, accompanied by quantitative evaluations, illustrates that many mechanisms are ill-suited to support the generation of disassembly ground truth. The mechanism best serving today’s need is to trace the compiling process of the target binaries to collect the ground truth information. Observing that the existing tool to trace the compiling process can still miss ground truth results and can only handle x86/x64 binaries, we extend the tool to avoid overlooking those results and support ARM32/AArch64/MIPS32/MIPS64 binaries. We envision that our extension will make the tool a better foundation to enable universal, standard ground truth for binary disassembly.
2023
(Abstract) Binary analyses based on deep neural networks (DNNs), or neural binary analyses (NBAs), have become a hotly researched topic in recent years. DNNs have been wildly successful at pushing the performance and accuracy envelopes in the natural language and image processing domains. Thus, DNNs are highly promising for solving binary analysis problems that are hard due to a lack of complete information resulting from the lossy compilation process. Despite this promise, it is unclear that the prevailing strategy of repurposing embeddings and model architectures originally developed for other problem domains is sound given the adversarial contexts under which binary analysis often operates. In this paper, we empirically demonstrate that the current state of the art in neural function boundary detection is vulnerable to both inadvertent and deliberate adversarial attacks. We proceed from the insight that current generation NBAs are built upon embeddings and model architectures intended to solve syntactic problems. We devise a simple, reproducible, and scalable black-box methodology for exploring the space of inadvertent attacks – instruction sequences that could be emitted by common compiler toolchains and configurations – that exploits this syntactic design focus. We then show that these inadvertent misclassifications can be exploited by an attacker, serving as the basis for a highly effective black-box adversarial example generation process. We evaluate this methodology against two state-of-the-art neural function boundary detectors: XDA and DeepDi. We conclude with an analysis of the evaluation data and recommendations for ho
(Abstract) Taint analysis has been widely used in many security applications such as exploit detection, information flow tracking, malware analysis, and protocol reverse engineering. State-of-theart taint analysis tools are usually built atop dynamic binary instrumentation, which instruments at every possible instruction, and rely on runtime information to decide whether a particular instruction involves taint or not, thereby usually having high performance overhead. This paper presents SELECTIVETAINT, an efficient selective taint analysis framework for binary executables. The key idea is to selectively instrument the instructions involving taint analysis using static binary rewriting instead of dynamic binary instrumentation. At a high level, SELECTIVETAINT statically scans taint sources of interest in the binary code, leverages value set analysis to conservatively determine whether an instruction operand needs to be tainted or not, and then selectively taints the instructions of interest. We have implemented SELECTIVETAINT and evaluated it with a set of binary programs including 16 coreutils (focusing on file I/O) and five network daemon programs (focusing on network I/O) such as nginx web server. Our evaluation results show that the binaries statically instrumented by SELECTIVETAINT has superior performance compared to the state-of-the-art dynamic taint analysis frameworks (e.g., 1.7x faster than that of libdft).