How Data-Driven Underwriting Reduces Bad Risk Selection

For underwriting, product, and analytics leaders, bad risk selection is rarely the result of carelessness. It is usually the result of incomplete information, premiums set without full visibility into the underlying risk, portfolios that look homogeneous in aggregate but carry hidden concentrations, and pricing models that were accurate when built but have since drifted. The consequences are familiar: loss ratio deterioration, adverse development, and the slow erosion of underwriting margin that only becomes obvious after the fact.

At Perceptive Analytics, we help carriers close the gap between the data they hold and the data they actually use in underwriting decisions.

What has changed materially in recent years is the breadth and quality of data available to underwriters, and the analytical methods capable of turning that data into underwriting decisions. The US P&C industry posted a combined ratio of 92.9% in 2025, its best in years, driven in part by pricing discipline and tighter underwriting. [Verisk / APCIA, 2025] But Verisk’s own analysis was clear: that improvement reflects rate action and a benign catastrophe year, not a fundamental shift in underlying risk management. The structural challenge of bad risk selection remains. This guide maps the data sources, analytical approaches, implementation realities, and governance requirements that underwriting leaders are using to address it, with the business case framing needed to move from concern to investment.

See our data-driven blueprint for insurance growth for the strategic context in which this guide sits.

Talk with our consultants today.

Is bad risk selection quietly eroding your underwriting margin? Let Perceptive Analytics show you what better data can do. Book a session with our experts now.

1. The Data That Matters Most for Accurate Underwriting

Most underwriting analytics programmes stall not on sophistication but on data. Before asking what models to build, the more important question is: what data do we actually have, and what are we still missing? The answer typically falls across four categories.

Internal Policy and Claims Data

The foundation. Historical loss experience by risk segment, line, geography, and vintage is the starting point for any predictive model. But internal data alone has a well-known limitation: it reflects the risks the carrier chose to write, not the full population. Adverse selection and survivorship bias are embedded in most internal datasets, which is why models trained purely on historical internal data tend to underestimate tail risk in segments the carrier has systematically underpriced.

• Loss and development history by cohort — the primary input for actuarial and ML pricing models.
• Policy characteristics at inception — exposure details, coverage elected, and application data, including any discrepancies between submitted information and verified facts.
• Renewal and lapse behaviour — profitable risks renew; adverse selection pressure concentrates in policies not renewed by well-rated competitors. Lapse patterns are an underused risk signal.
• Claims adjuster notes and free-text fields — unstructured internal data that, when processed through NLP, surfaces underwriting quality issues invisible in structured records.

External and Third-Party Data

This is where the analytical edge is being built. Carriers that integrate external data sources consistently achieve more precise risk segmentation. A 2024 Willis Towers Watson study found that P&C insurers implementing predictive modelling with enriched external data experienced a 67% improvement in risk assessment accuracy and a 5.7% decrease in combined ratios [Willis Towers Watson / Decerto, 2024], with premium leakage reduced by approximately $14 million per billion dollars of written premium. Perceptive Analytics’ advanced analytics consultants help carriers identify and integrate the external data that will move the needle most in their specific lines.

• Credit and financial data – correlated with loss frequency in personal and small commercial lines. Subject to regulatory constraints (discussed in Section 4).
• Telematics and IoT data – in personal auto, telematics programmes now cover more than 21 million US policyholders [IoT Insurance Observatory, 2024], enabling behaviour-based pricing that replaces broad demographic proxies. The global insurance telematics market was valued at $6.8 billion in 2024 and is growing at a CAGR of 18.9%. [GM Insights, 2025]
• Geospatial and climate risk data – property-level flood, wildfire, subsidence, and convective storm risk scores that allow underwriters to price and select at address level rather than postcode or zip code.
• Third-party risk scores and industry databases – ISO ClaimSearch, motor vehicle records, prescription history databases (for life and health), and commercial credit and financial health indicators for commercial lines.
• Satellite and aerial imagery – machine vision analysis of roof condition, property maintenance, vegetation proximity, and surrounding hazards, enabling property inspections at scale without physical surveys.

Unstructured Data

Natural language processing has made unstructured data tractable for underwriting. Application notes, inspection reports, broker submissions, and even publicly available information can be processed at volume to surface risk signals that structured data misses. By 2027, over 80% of new insurance policies are projected to be underwritten with some form of advanced analytics input [World Insurance Report 2024] – and much of that analytics will draw on previously unused unstructured sources.

Perceptive Analytics’ AI consulting team deploys NLP pipelines for submission analysis, inspection report processing, and broker note extraction.

2. How Leading Insurers Use Analytics To Reduce Loss Ratios

The narrative that analytics investment is speculative or unproven no longer holds. Carriers using predictive underwriting models report 15-25% improvements in loss ratios within the first 18 months [Capgemini World P&C Insurance Report / Vantage Point, 2026], and 83% of insurance executives now consider predictive models critical for underwriting’s future. [Capgemini World P&C Insurance Report] Three anonymised case snapshots illustrate what that looks like in practice.

Perceptive Analytics helps clients achieve these results through our Power BI consulting, Tableau consulting, and AI consulting practices.

Personal Lines: Behaviour-Based Segmentation in Auto

A large personal auto carrier was experiencing loss ratio deterioration in its urban book, with claims severity rising faster than rate. The root cause, identified through telematics data and geospatial analysis, was concentration risk in specific zip codes where distracted driving rates, road condition scores, and historical claims frequency combined to create a materially higher risk profile than the carrier’s rating plan reflected.

Perceptive Analytics’ price optimisation analysis and marketing analytics practices deliver for personal lines carriers.

By integrating telematics-derived driving behaviour scores into the renewal rating algorithm and adjusting pricing by micro-geography, the carrier achieved a 4-point improvement in its urban book loss ratio within 12 months. Critically, the change also improved retention of preferred-risk customers, who were previously subsidising adverse risks in the same pricing tier — a double benefit from better segmentation.

Commercial Lines: Predictive Risk Scoring in Small Business

A mid-market commercial lines carrier writing small business property and liability was seeing above-expectation frequency in its hospitality segment. Traditional underwriting relied on SIC code, revenue, and years in business. A pilot using external financial health scores, payment behaviour data, and geographic hazard overlays produced a risk ranking model that identified a 30% subset of the hospitality book carrying 60% of the loss exposure.

The carrier used the model outputs to tighten appetite in the highest-risk deciles while maintaining competitive pricing for lower-risk accounts. Combined ratio in the segment improved by 6 points over 18 months, driven primarily by adverse risk exits and pricing adjustment, not rate increases applied across the board.

Property: Geospatial Data Replacing Physical Inspections

A regional property insurer writing homeowners in climate-exposed states was finding that physical inspection costs were limiting the granularity of risk selection in its lower-value property book. By deploying satellite-based machine vision to assess roof condition, tree proximity, and maintenance indicators at address level, an approach now offered by several geospatial data providers at scale, the carrier was able to underwrite with the precision previously reserved for high-value properties across its entire book.

The result was a reduction in severe property claims from poorly maintained risks that had previously been accepted without detailed inspection, and a 20% reduction in physical inspection costs. The net combined ratio impact was a 3-point improvement, predominantly in the loss ratio, within the first year of deployment.

3. Common Pitfalls When Embedding Analytics in Underwriting

The ROI case for underwriting analytics is compelling. The implementation record is more mixed. The gap between the two is not usually a technology problem; it is a combination of data quality issues, organisational change resistance, and model governance gaps that compound over time if not addressed upfront.

Data Quality and Completeness

Predictive models are only as reliable as the data used to train them. Incomplete policy data, inconsistently coded claim records, and manual workarounds that create silent data gaps are endemic in carriers with legacy policy admin systems. Siloed data is consistently cited as the primary barrier to effective insurance analytics. [Vantage Point / Salesforce, 2026] A model trained on incomplete or biased historical data will reproduce and amplify the biases embedded in that data, including the adverse selection patterns the carrier is trying to escape. 

The mitigation is not to wait for perfect data, which never arrives, but to implement a data quality programme in parallel with model development, with explicit documentation of known data gaps and their potential impact on model outputs. Data quality remediation is often the highest-ROI investment in an analytics programme, but it is consistently underfunded relative to model development.

Change Management and Underwriter Adoption

Underwriting analytics programmes fail most often not at the model stage but at the workflow integration stage. Underwriters who receive a risk score without understanding how it was generated, what it represents, and how to use it alongside their own judgement will either ignore it or override it inconsistently. While 88% of commercial fleets have telematics programmes, only 64% of carriers actually use that data in underwriting decisions [Carrier Management / Indenseo, 2025], the data exists; the workflow connection does not.

Effective change management means embedding analytics outputs directly into underwriting workflows, training underwriters on interpretation, and creating feedback mechanisms so that overrides are captured, reviewed, and fed back into model improvement. The goal is a culture where analytics augments underwriter judgement rather than competing with it.

Perceptive Analytics embeds analytics outputs directly into underwriting workflows using Tableau development services and Power BI development services, and provides training frameworks that build underwriter confidence in model outputs.

Model Governance and Performance Drift

Models built on historical data degrade over time as the risk environment evolves. A model trained before the pandemic will not accurately reflect post-pandemic driving behaviour. A commercial property model trained before climate acceleration will underestimate emerging concentration risk. Without a structured model governance programme, regular performance monitoring, champion-challenger testing, and scheduled recalibration, analytics investments that delivered returns in year one can become liabilities by year three.

See our data transformation maturity guide for the governance framework Perceptive Analytics recommends.

Legacy System Integration

The largest single implementation cost in most underwriting analytics deployments is system integration, not software licensing. Connecting an analytics layer to legacy policy admin, claims, and data warehouse systems that were not designed for API integration requires significant IT investment and time. Accenture’s 2024 Insurance Technology Vision report found that insurers implementing AI-driven workflow automation reduced policy processing times by 50–70% [Accenture, 2024], but achieving those outcomes requires the integration infrastructure to be in place first.

4. Using Data Responsibly: Regulatory and Compliance Considerations

Regulatory frameworks governing the use of data and AI in underwriting are moving faster than most insurers anticipated. The direction of travel is clear: greater transparency, explainability requirements, and demonstrated evidence that models do not produce unfairly discriminatory outcomes. The carriers that treat compliance as a design constraint, built into model development from the start, will navigate this environment with significantly less friction than those treating it as a retrofit.

The NAIC Framework and State-Level Adoption

The NAIC adopted its Model Bulletin on the Use of Artificial Intelligence Systems by Insurers in December 2023, establishing the de facto national standard for AI governance in insurance. As of 2025, 24 states have fully adopted the Model Bulletin [Baker Tilly, 2025], with additional states implementing their own guidelines. The Bulletin requires insurers to implement, maintain, and document an AI Systems (AIS) Programme covering governance, risk management, validation, testing, and bias monitoring.

In parallel, the NAIC’s Big Data and Artificial Intelligence Working Group is developing an AI Systems Evaluation Tool, currently being piloted by 12 states, that regulators will use in market conduct examinations to assess how insurers use AI in underwriting, pricing, and claims. [NAIC, March 2026] Of the 193 auto insurers surveyed by the NAIC, 88% reported using, planning to use, or planning to explore AI/ML models in their operations [NAIC AI Survey, 2024], underscoring the regulatory scrutiny that volume of adoption is attracting. Our choosing a trusted Tableau partner for data governance guide outlines how we approach governance in BI and analytics deployments.

Key Regulatory Requirements for Underwriting Analytics

• Fairness and non-discrimination: AI underwriting decisions must comply with the Unfair Trade Practices Act. Models that produce disparate impacts on protected classes, even without explicit use of protected characteristics, are subject to regulatory challenge. New York DFS Circular Letter 2024-7 specifically requires insurers to demonstrate that AI and external data systems do not proxy for protected classes. [Buchanan Ingersoll & Rooney, 2025]
• Explainability and adverse action: Where AI or predictive models contribute to adverse underwriting decisions, carriers must be able to provide consumers with specific, meaningful reasons. Black-box models that cannot generate explainable outputs are increasingly difficult to defend in regulatory examinations.
• Third-party vendor oversight: Insurers remain accountable for the outputs of third-party data and model providers. The NAIC’s Third-Party Data and Models Working Group is developing a framework for regulatory oversight of these relationships — carriers should ensure contractual audit rights and data transparency from all analytics vendors.
• Data privacy: The use of external consumer data — particularly telematics, social media, and behavioural data — is subject to state privacy laws (CCPA and its successors), HIPAA where health data is involved, and evolving federal guidance on AI-driven consumer decisions from the CFPB.

The compliance investment is real. But a 2024 KPMG compliance survey found that insurers with formal review processes for analytics models experienced 75% fewer regulatory challenges [KPMG, 2024 / Decerto] compared to those without them. Governance is not just a compliance cost, it is a risk management lever.

5. Cost and ROI of Data-Driven Underwriting

The business case for underwriting analytics investment is not difficult to make in principle. The challenge is making it with enough specificity to compete for capital internally. That requires a clear view of cost categories, realistic benefit assumptions, and a payback framing that is defensible to a CFO or board.

Cost Categories

• Data acquisition: Third-party data licences for credit, geospatial, telematics, and specialty risk scores. Costs range from a few cents per policy for basic credit scores to several dollars per address for high-resolution climate and geospatial data. Volume discounts are significant — this cost is often lower at scale than pilots suggest.
• Analytics platform and infrastructure: The insurance big data analytics market was valued at $12.3 billion in 2025 and is growing at 13.4% CAGR through 2033. [Vantage Point, 2026] Platform costs vary significantly by architecture choice — SaaS solutions lower upfront costs but carry ongoing subscription expense; on-premise or cloud-native builds require higher initial investment but offer greater customisation.
• Integration and implementation: Typically the largest single cost driver. Connecting analytics to legacy policy admin and data infrastructure is expensive and time-consuming. Analytics pilots cost as little as $50,000 for a focused use case; enterprise-wide data platform implementations run $500,000 or more. [Vantage Point, 2026]
• Talent: Data scientists, actuaries with analytics expertise, and ML engineers. The market for this talent is competitive; partnerships with analytics vendors or specialist consulting firms are often more cost-effective than pure in-house builds for mid-sized carriers.
• Ongoing governance and model maintenance: Model recalibration, bias monitoring, documentation updates, and regulatory examination readiness. Consistently underbudgeted in initial business cases, carriers should allocate 15–20% of the initial build cost annually for ongoing model maintenance.

The ROI Framework

The benefit side of the equation is anchored in three levers. Premium leakage reduction is the most direct: better risk identification prevents underpriced risk from entering the portfolio. The WTW study cited earlier quantified this at $14 million per billion dollars of written premium, a tangible and auditable figure. [Willis Towers Watson / Decerto, 2024]

Loss ratio improvement through better risk selection and pricing adequacy. Carriers using predictive underwriting models report 15–25% loss ratio improvements within 18 months, [Capgemini / Vantage Point] with a 5.7% decrease in combined ratios in documented implementations. [Willis Towers Watson, 2024] For a carrier with $1 billion in gross written premium operating at a 98% combined ratio, a 3-point combined ratio improvement represents $30 million in underwriting gain, transformative at typical IT investment levels.

Operational efficiency from straight-through processing and automated triage. Accenture documented 50-70% reductions in policy processing times [Accenture, 2024] at carriers implementing AI-driven workflow automation, alongside 30% reductions in administrative costs. These savings offset a portion of the technology cost and free underwriters to focus on complex risks where judgement creates the most value.

Goldman Sachs’s 2024 Global Insurance Survey found that only 29% of insurance companies globally use AI in underwriting [Goldman Sachs, 2024], meaning the majority of the industry is still operating without the analytical advantage that data-driven underwriting provides. For carriers willing to invest now, the competitive window remains genuinely open.

6. Practical Next Steps To Modernize Your Underwriting Analytics

Modernising underwriting analytics is a multi-year programme, but the carriers that make meaningful progress start with a focused, sequenced set of actions rather than attempting enterprise-wide transformation in parallel. The following steps reflect how leading carriers have translated aspiration into measurable results.

Step 1: Audit Your Current Data Estate

Before any model work begins, map what data you have, where it lives, and what condition it is in. Identify the highest-value gaps, the external data sources that would most significantly improve risk selection in your priority lines, and quantify the exposure concentration you currently cannot see. This diagnostic typically surfaces the 2-3 data investments that will drive 80% of the analytical improvement.

Step 2: Run a Bounded Pilot on a High-Loss Line

Select the line of business with the highest fraud intensity, most significant adverse selection pressure, or largest gap between quoted and actual loss experience. Deploy an analytics pilot, whether using internal ML resource, a specialist vendor, or a data partner, that produces a ranked risk score for a defined book segment. Compare the score distribution against actual loss experience. This exercise almost always surfaces the concentration the carrier suspected but could not previously quantify.

Step 3: Embed, Don’t Just Report

Analytics that remain in a dashboard or reporting layer do not move underwriting decisions. The score or signal needs to surface in the underwriting workflow, visible to the underwriter at the point of decision, integrated into submission triage, or applied automatically in straight-through processing for low-complexity risks. The integration design matters as much as the model quality.

Step 4: Build Governance from the Start

Establish the AI Systems Programme required under the NAIC Model Bulletin before models go live. Document model inputs, validation methodology, bias testing results, and ongoing monitoring protocols. [NAIC Model Bulletin, December 2023] A governance-first approach is not a compliance overhead — it is the architecture that allows models to be trusted, scaled, and defended under regulatory scrutiny.

Step 5: Measure Against a Clear Baseline

Define the KPIs before the pilot begins. Loss ratio by cohort, premium leakage rate, quote-to-bind improvement in targeted risk segments, time-to-decision, and SIU referral rate are the metrics that capture the business impact of better underwriting. Without a pre-defined baseline, it is impossible to demonstrate the return on investment with the specificity that boards and CFOs require.

Step 6: Evaluate Partners and Solutions on Integration Capability, Not Just Model Performance

The most common mistake in vendor selection is prioritising model sophistication over integration track record. A model that cannot connect to your policy admin system, that requires significant data engineering to operationalise, or that lacks explainable output capability is not a solution, it is a prototype. Evaluate vendors on their ability to deploy in your technical environment, their experience with carriers of your size and line mix, and their regulatory documentation standards.

Download our underwriting analytics ROI checklist to quantify the premium leakage and loss ratio opportunity in your current book, or request a tailored underwriting analytics assessment to identify the highest-value data and modelling investments for your specific lines and portfolio profile.

Final Perspective: Data Is Not an Advantage If You Do Not Use It

The insurance industry has never had access to more risk-relevant data than it does today — telematics signals from 21 million connected vehicles, address-level climate and geospatial risk scores, real-time financial health indicators for commercial risks, and machine vision analysis that can inspect a property without a physical survey. The analytical methods to process this data into underwriting decisions are mature, proven, and increasingly accessible.

Perceptive Analytics exists to help carriers close that gap — through our AI consulting, advanced analytics, Power BI implementation, and Tableau implementation capabilities.

And yet, only 29% of insurance companies globally [Goldman Sachs, 2024] use AI in their underwriting operations. The gap between data availability and data utilisation is where the next wave of underwriting advantage will be built, and where the next wave of underwriting loss for carriers that move slowly will accumulate.

For mid-sized carriers in particular, this is not a technology problem. It is a prioritisation and sequencing problem. The data is available. The vendors are ready. The regulatory framework, while evolving, is navigable for carriers that build governance into their approach from the start. The carriers that begin now, with a diagnostic, a bounded pilot, and a clear measurement framework, are the ones that will be explaining their superior combined ratios in five years rather than trying to explain their deteriorating ones.

The question is not whether data-driven underwriting works. The question is how long your current approach will remain competitively adequate.

Talk with our consultants today.

Ready to close the gap between the data you have and the underwriting decisions you make? Perceptive Analytics is here to help. Book a session with our experts now.

Sources & References

[1] Verisk / APCIA (2025). Strong 2025 Underwriting Income Masks Persistent P/C Insurance Pressures.

[2] Willis Towers Watson / Decerto (2024). Insurance Software with Predictive Analytics: A Competitive Edge.

[3] IoT Insurance Observatory / Arity (2024). Telematics and Trust.

[4] GM Insights (2025). Insurance Telematics Market Size, Growth Forecasts 2025–2034.

[5] World Insurance Report 2024. Prediction on analytics-underwritten policies by 2027.

[6] Capgemini World Property and Casualty Insurance Report. 83% of executives believe predictive models are critical.

[7] Vantage Point (March 2026). Data Analytics for Insurance: A Strategic Guide.

[8] S&P Global Market Intelligence (2025). US P&C industry net combined ratio 96.5% in 2024.

[9] Carrier Management / Indenseo (November 2025). Why Insurance Telematics Integrations Fail.

[10] Accenture (2024). Insurance Technology Vision Report.

[11] NAIC (December 2023). Model Bulletin: Use of Artificial Intelligence Systems by Insurers.

[12] NAIC Big Data and Artificial Intelligence (H) Working Group (March 2026). AI Systems Evaluation Tool.

[13] NAIC AI Survey (2024). 88% of auto insurers use or plan to use AI/ML.

[14] Baker Tilly (August 2025). The Regulatory Implications of AI and ML for the Insurance Industry.

[15] Buchanan Ingersoll & Rooney (October 2025). When Algorithms Underwrite: NY DFS Circular Letter 2024-7.

[16] KPMG (2024). Compliance survey — 75% fewer regulatory challenges with formal model review.

[17] Goldman Sachs (2024). Global Insurance Survey — only 29% of insurance companies use AI globally.

[18] Gartner (2023). Insurance Technology Survey.

[19] Accenture (2024). 30% reduction in administrative costs from AI-driven automation.

[20] AM Best (2025). US personal auto decade-high net underwriting profit.

[21] McKinsey & Company (2021). How Data and Analytics Are Redefining Excellence in P&C Underwriting.