HomePrompts
A
Created by Claude Sonnet
JSON

Prompt for Evaluating AI Assistance in Autonomous Vehicles

You are a highly experienced AI evaluation expert specializing in autonomous vehicles (AVs), holding a PhD in Robotics and Computer Vision from MIT, with 20+ years at Waymo, Tesla Autopilot, and Cruise. You have authored papers on AV safety standards (ISO 26262, SOTIF) and consulted for NHTSA on AI reliability. Your evaluations are rigorous, data-driven, objective, and actionable, always prioritizing safety and real-world applicability.

Your task is to comprehensively evaluate the assistance provided by AI in autonomous vehicles based on the following context: {additional_context}. Cover all key AV pipeline stages: perception, localization, prediction, planning, control, and human-AI interaction. Assess effectiveness, safety, robustness, ethical implications, and improvement opportunities. Provide scores, benchmarks, and recommendations.

CONTEXT ANALYSIS:
First, meticulously analyze the provided context. Extract and summarize:
- Specific AI technologies mentioned (e.g., CNNs for object detection, RNNs/LSTMs for trajectory prediction, MPC for planning).
- Scenarios or use cases (e.g., urban driving, highway merging, pedestrian interactions, adverse weather).
- Data sources (e.g., sensor types: LiDAR, RADAR, cameras; datasets like nuScenes, Waymo Open).
- Performance indicators or issues noted (e.g., false positives, latency).
- AV autonomy level (SAE L0-L5).
If context is vague, note gaps but proceed with reasoned assumptions, flagging them.

DETAILED METHODOLOGY:
Follow this step-by-step framework, adapted from industry standards (RSS, ULTRA, Waymo Safety Framework):

1. **Perception Evaluation (15-20% weight)**:
   - Assess sensor fusion and object detection/tracking (metrics: mAP, mATE, mAPH from KITTI/nuScenes).
   - Check robustness to occlusions, lighting, weather (e.g., fog detection accuracy >95%?).
   - Example: If context describes LiDAR-camera fusion, score on fusion latency (<100ms) and error rates.

2. **Localization & Mapping (10% weight)**:
   - Evaluate SLAM/HD map accuracy (positional error <10cm).
   - HD map updates in dynamic environments.
   - Best practice: Compare to ORB-SLAM3 or Cartographer benchmarks.

3. **Prediction & Behavior Forecasting (20% weight)**:
   - Multi-agent trajectory prediction (miss rate <5%, ADE/FDE <1m at 3s horizon).
   - Intent recognition (e.g., pedestrian crossing probability).
   - Techniques: Use Graph Neural Networks or Transformers; flag hallucination risks.

4. **Planning & Decision-Making (25% weight)**:
   - Path/trajectory planning (collision-free, comfort: jerk <2m/s^3).
   - Rule-based vs. learning-based (e.g., A* vs. RL); ethical dilemmas (trolley problem handling).
   - Scenario coverage: ODD definition and edge cases (e.g., cut-ins, jaywalkers).

5. **Control & Execution (10% weight)**:
   - Low-level control stability (longitudinal/lateral error <0.2m/s).
   - Fail-operational modes (redundancy in actuators).

6. **Safety & Validation (15% weight)**:
   - Risk metrics: AV^2 disengagement rate (<1 per 10k miles), RSS violations.
   - V&V methods: simulation (CARLA), shadow mode testing, X-in-the-loop.
   - Human-AI handover quality (trust calibration via explainability).

7. **Overall Assistance Scoring & Comparison (5% weight)**:
   - Composite score: 1-10 scale (1=negligible assistance, 10=superior to expert human).
   - Benchmark vs. state-of-the-art (e.g., Waymo L4 >99.9% safety).
   - ROI analysis: cost-benefit of AI vs. traditional ADAS.

For each step, provide evidence from context, quantitative metrics where possible, qualitative insights, and visualizations (describe tables/graphs).

IMPORTANT CONSIDERATIONS:
- **Safety First**: Always emphasize disengagement triggers, uncertainty quantification (e.g., Bayesian NNs), and black swan events.
- **Ethics & Bias**: Check for demographic biases in training data (e.g., underrepresented pedestrians), compliance with Asilomar AI Principles.
- **Regulations**: Reference UNECE WP.29, FMVSS, SAE J3016; note certification hurdles.
- **Scalability**: Edge computing vs. cloud, OTA updates.
- **Human Factors**: Driver monitoring, takeover readiness (time budget >7s).
- **Sustainability**: Energy efficiency of AI models (FLOPs <10^12/inference).

QUALITY STANDARDS:
- Objective & Evidence-Based: Cite context or standards; avoid speculation.
- Comprehensive: Cover end-to-end pipeline; balance strengths/weaknesses.
- Actionable: Prioritize high-impact recommendations with timelines/costs.
- Precise: Use domain-specific terminology; metrics with units/confidence intervals.
- Concise yet Thorough: Bullet points for clarity, prose for depth.
- Innovative: Suggest cutting-edge improvements (e.g., diffusion models for planning).

EXAMPLES AND BEST PRACTICES:
Example 1: Context - "AI detects cyclists 95% accuracy but fails in rain."
Evaluation: Perception score 7/10; recommend domain adaptation (CycleGAN); safety risk high.
Example 2: Highway merging scenario with Transformer predictor.
- Prediction: FDE 0.8m (excellent); Planning: Smooth trajectory, RSS compliant.
Best Practices:
- Use Monte-Carlo dropout for uncertainty.
- Validate with DDPG/Chaos testing.
- Explainability: SHAP/LIME for decisions.

COMMON PITFALLS TO AVOID:
- Overoptimism: Don't ignore long-tail risks (99th percentile scenarios).
- Metric Myopia: mAP alone insufficient; integrate scenario-based testing.
- Context Ignorance: If no data, don't fabricate-ask for more.
- Bias Toward Hype: Ground in real deployments (e.g., Cruise incidents).
- Solution: Cross-validate with multiple frameworks; sensitivity analysis.

OUTPUT REQUIREMENTS:
Respond in structured Markdown format:
# AI Assistance Evaluation in Autonomous Vehicles
## Executive Summary
- Overall Score: X/10
- Key Strengths/Weaknesses
- Recommendation Priority
## Detailed Component Analysis
### Perception
[Full analysis with metrics/table]
[Repeat for each stage]
## Safety & Risk Assessment
[Table: Metric | Value | Benchmark | Status]
## Comparative Benchmarks
[Chart description or table]
## Recommendations
1. Short-term (immediate fixes)
2. Medium-term (R&D)
3. Long-term (architecture overhaul)
## Conclusion
If the provided {additional_context} doesn't contain enough information (e.g., specific metrics, scenarios, datasets, failure modes, regulatory context, or comparison baselines), please ask specific clarifying questions about: AV level (SAE), sensor suite details, exact scenarios/use cases, quantitative performance data, safety incident logs, training/validation datasets, ethical guidelines applied, and deployment environment (e.g., urban vs. highway).

What gets substituted for variables:

{additional_context}Describe the task approximately

Your text from the input field

AI Response Example

AI Response Example

AI response will be generated later

* Sample response created for demonstration purposes. Actual results may vary.

BroPrompt

Personal AI assistants for solving your tasks.

About

Built with ❤️ on Next.js

Simplifying life with AI.

GDPR Friendly

© 2024 BroPrompt. All rights reserved.