|
| 1 | +--- |
| 2 | +# Documentation: https://wowchemy.com/docs/managing-content/ |
| 3 | + |
| 4 | +title: "Seminar: \"SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation\"" |
| 5 | +# event: |
| 6 | +# event_url: |
| 7 | +location: Abacws |
| 8 | +# address: |
| 9 | +# street: |
| 10 | +# city: |
| 11 | +# region: |
| 12 | +# postcode: |
| 13 | +# country: |
| 14 | +summary: Talk by [Jingxuan Chen](https://www.linkedin.com/in/jingxuanchen916/) (Huawei) |
| 15 | +abstract: "Smartphone control agents, based on (Multimodal) Large Language Models, operate smartphones in a human-like manner by observing the screen and performing actions such as tapping or typing. These agents hold great promise for assisting users with everyday tasks, from setting alarms to booking hotels. In this talk, I will present SPA-Bench, our newly proposed benchmark for systematic smartphone agent evaluation, accepted as an ICLR 2025 Spotlight. SPA-Bench supports agent interaction with Android devices across a wide range of realistic tasks in both English and Chinese, and features a scalable, automated evaluation pipeline. I will share insights from evaluating eleven existing trending agents and discuss how SPA-Bench helps identify their strengths and limitations, offering a foundation for future research and real-world deployment." |
| 16 | + |
| 17 | +# Talk start and end times. |
| 18 | +# End time can optionally be hidden by prefixing the line with `#`. |
| 19 | +date: 2025-03-27T13:00:00Z |
| 20 | +date_end: 2025-03-27T14:00:00Z |
| 21 | +all_day: false |
| 22 | + |
| 23 | +# Schedule page publish date (NOT event date). |
| 24 | +publishDate: 2025-03-26T00:00:00Z |
| 25 | + |
| 26 | +authors: [ousidhoumn] |
| 27 | +tags: [] |
| 28 | + |
| 29 | +# Is this a featured event? (true/false) |
| 30 | +featured: false |
| 31 | + |
| 32 | +# Featured image |
| 33 | +# To use, add an image named `featured.jpg/png` to your page's folder. |
| 34 | +# Focal points: Smart, Center, TopLeft, Top, TopRight, Left, Right, BottomLeft, Bottom, BottomRight. |
| 35 | +image: |
| 36 | + caption: "" |
| 37 | + focal_point: "" |
| 38 | + preview_only: false |
| 39 | + |
| 40 | +# Custom links (optional). |
| 41 | +# Uncomment and edit lines below to show custom links. |
| 42 | +# links: |
| 43 | +# - name: Follow |
| 44 | +# url: https://twitter.com |
| 45 | +# icon_pack: fab |
| 46 | +# icon: twitter |
| 47 | + |
| 48 | +# Optional filename of your slides within your event's folder or a URL. |
| 49 | +url_slides: |
| 50 | + |
| 51 | +url_code: |
| 52 | +url_pdf: |
| 53 | +url_video: |
| 54 | + |
| 55 | +# Markdown Slides (optional). |
| 56 | +# Associate this event with Markdown slides. |
| 57 | +# Simply enter your slide deck's filename without extension. |
| 58 | +# E.g. `slides = "example-slides"` references `content/slides/example-slides.md`. |
| 59 | +# Otherwise, set `slides = ""`. |
| 60 | +slides: "" |
| 61 | + |
| 62 | +# Projects (optional). |
| 63 | +# Associate this post with one or more of your projects. |
| 64 | +# Simply enter your project's folder or file name without extension. |
| 65 | +# E.g. `projects = ["internal-project"]` references `content/project/deep-learning/index.md`. |
| 66 | +# Otherwise, set `projects = []`. |
| 67 | +projects: [] |
| 68 | +--- |
| 69 | + |
| 70 | +**Invited Speaker:** [Jingxuan Chen](https://www.linkedin.com/in/jingxuanchen916/) (Huawei) |
0 commit comments