36,769 Self-Hosted AI Services Exposed Online — What Security Teams Should Check

Self-hosted AI promises tighter control over sensitive workloads. A new internet-wide scan found tens of thousands of those systems reachable from the public internet.

Researchers identified 36,769 self-hosted AI endpoints across model servers, workflow tools, and vector stores, according to research published Sept. 10 by Mysterium VPN. Only 741, or 2.02%, returned an HTTP authentication challenge.

That does not mean the remaining 98% lacked an application login. A service can return HTTP 200 while still presenting its own sign-in page. The Mysterium VPN research found that only a small fraction showed the network- or HTTP-layer authentication gate its methodology could detect.

Exposure extends beyond model servers

Open WebUI accounted for 18,529 reachable endpoints, followed by 6,935 Ollama servers and 4,880 vLLM endpoints. Researchers also identified 5,223 endpoints associated with AI workflow and agent-building platforms, plus 920 vector-store endpoints they cautioned were likely undercounted.

Workflow platforms warrant particular scrutiny because they can connect AI models to APIs, databases, cloud services, webhooks, and credentials. Those connections also make trust and access controls for AI agents important beyond the model itself.

Ollama stood out because 6,935 servers returned its identifying root response to an anonymous request. Ollama documentation confirms that authentication is not required when accessing its API locally.

Public AI infrastructure is also drawing active attacks. Recent attacks on LiteLLM and MCP servers involved credential theft, command execution, cryptomining, and attempts to reach resources connected to AI services. In a separate 72-hour instrumented deployment, CyberDesserts researchers observed AI-specific reconnaissance and searches for credentials and configuration files but did not establish widespread exploitation of AI models themselves.

Lock down the AI attack surface

Organizations should first determine which AI services are externally reachable and whether that exposure is intentional. Controls should then cover network access, credentials, software hygiene, monitoring, and response.

• Inventory public-facing AI services. Include model servers, web interfaces, agent platforms, and vector databases in attack-surface management and external exposure scans.
• Restrict unnecessary network access. Keep services private where possible, use firewall rules or IP allowlists, and protect required remote access with authenticated proxies, VPNs, or zero-trust controls.
• Patch and harden deployments. Keep AI software and supporting components current. CVE-2024-37032, known as Probllama, affected Ollama versions before 0.1.34 and was fixed in version 0.1.34. CISA has also added actively exploited AI infrastructure flaws to its Known Exploited Vulnerabilities catalog.
• Protect and rotate credentials. Review exposed workflow systems for API keys, cloud tokens, database credentials, and webhook secrets, then rotate potentially exposed secrets and apply least privilege.
• Limit abuse and lateral movement. Apply rate limits, segment AI infrastructure from sensitive systems, and restrict unnecessary outbound connections.
• Monitor for suspicious activity. Review proxy, firewall, cloud-flow, and application logs for unusual source IPs, abnormal request volumes, unexpected inference activity, and other API anomalies.
• Test incident response plans for AI exposure. Exercise containment, log preservation, credential rotation, historical access review, and investigation of connected systems.

These measures are consistent with NIST SP 800-228, which maps API risks and controls across development and runtime stages. The 36,769 endpoints are not 36,769 confirmed compromises; they show how much self-hosted AI infrastructure is visible beyond private environments.

Reducing that exposure, limiting privileges, and monitoring access can keep a reachable service from becoming a wider incident.

Read more: A recent NemoClaw vulnerability demonstrated how an exposed Ollama backend could let a malicious website alter the model used by an AI agent, underscoring why local inference servers should be treated as privileged infrastructure.