3 results
Search Results
Now showing 1 - 3 of 3
Article A Model-Based Evaluation Metric for Question Answering Systems(World Scientific, 2025) Baklr, D.; Aktas, M.S.; Ylldlz, B.The paper addresses the limitations of traditional evaluation metrics for Question Answering (QA) systems that primarily focus on syntax and n-gram similarity. We propose a novel model-based evaluation metric, MQA-metric, and create a human-judgment-based dataset, squad-qametric and marco-qametric, to validate our approach. The research aims to solve several key problems: the objectivity in dataset labeling, the effectiveness of metrics when there is no syntax similarity, the impact of answer length on metric performance, and the influence of real answer quality on metric results. To tackle these challenges, we designed an interface for dataset labeling and conducted extensive experiments with human reviewers. Our analysis shows that the MQA-metric outperforms traditional metrics like BLEU, ROUGE and METEOR. Unlike existing metrics, MQA-metric leverages semantic comprehension through large language models (LLMs), enabling it to capture contextual nuances and synonymous expressions more effectively. This approach sets a standard for evaluating QA systems by prioritizing semantic accuracy over surface-level similarities. The proposed metric correlates better with human judgment, making it a more reliable tool for evaluating QA systems. Our contributions include the development of a robust evaluation workflow, creation of high-quality datasets, and an extensive comparison with existing evaluation methods. The results indicate that our model-based approach provides a significant improvement in assessing the quality of QA systems, which is crucial for their practical application and trustworthiness. © 2025 World Scientific Publishing Company.Article Citation - WoS: 1Citation - Scopus: 1Model Uncertainty and Financial Frictions: Implications for Optimal Monetary Policy(World Scientific, 2024) Kantur,Z.; Özcan,G.The last decades proved that policymaking without considering uncertainty is impracticable. In an environment of uncertainty, policymakers have doubts about the policy models they routinely use. This paper focuses specifically on the situation where uncertainty on the financial side of the economy leads to misspecification in the policy model. We describe a coherent strategy for policymakers who are averse to model misspecification and analyze optimal policy design in the face of Knightian uncertainty. To do so, we augment a financial dynamic stochastic general equilibrium model with model misspecification in a simple minimax framework where the central bank plays a zero-sum game versus a hypothetical evil agent. The policy is tailored to insure against the worstcase outcomes. We show that model ambiguity on the financial side requires a passive monetary policy stance. However, if the uncertainty originates from the supply side of the economy, an aggressive response of interest rate is required. We also show the impact of an additional macroprudential tool on the dynamics of the economy. © 2024 World Scientific Publishing Company.Article Variations on a Theme of Mirsky(World Scientific, 2023) Akbal,Y.; Güloǧlu,A.M.Let k and r be non-zero integers with r ≥ 2. An integer is called r-free if it is not divisible by the rth power of a prime. A result of Mirsky states that there are infinitely many primes p such that p + k is r-free. In this paper, we study an additive Goldbach-type problem and prove two uniform distribution results using these primes. We also study certain properties of primes p such that p + a1,...,p + aℓ are simultaneously r-free, where a1,...,aℓ are non-zero integers and ℓ ≥ 1. © 2023 World Scientific Publishing Company.
