Clicky
github.com/farzaa/clicky- Category
- AI Agents
- Rank
- No. 428Tools index
- Pricing
- Open Source
- Type
- APP
- Builder
- @FarzaTV
- GitHub
- 7.5k stars
- Date
About
An AI teacher that lives next to your cursor on Mac, can see your screen, and provides real-time guidance by talking to you and pointing at UI elements. Like having a personal tutor watching over your shoulder.
What it does
Clicky is a macOS menu bar companion operated through a push-to-talk shortcut. It captures microphone input, takes screenshots during hotkey use, sends the transcript and images to a language model, streams the reply into its interface, speaks it aloud, and can animate toward identified controls across multiple monitors.
Why it's ranked here
The interaction loop is unusually concrete: speak, capture context, receive streamed guidance, hear the answer, and see the relevant control indicated. The open-source edition also exposes enough Swift code to modify the model, overlay, transcription, and speech pipeline. Setup is substantial, and the newest development continues privately.
What's good
API keys stay behind a Cloudflare Worker instead of shipping inside the application binary. The Claude client supports multiple labeled images, conversation history, progressive text updates, PNG and JPEG detection, and explicit HTTP error reporting. Its transparent overlays ignore mouse events and avoid stealing focus while operating across spaces and multiple displays.
Tradeoffs
Local setup requires recent macOS and Xcode, Node.js, a Cloudflare account, three separate API credentials, deployed secrets, a voice identifier, signing configuration, and manual replacement of hardcoded proxy addresses. Users must grant microphone, accessibility, screen recording, and screen content permissions. Analytics records transcripts and AI responses through PostHog, while onboarding also submits email addresses to FormSpark.
How to use it well
It best suits Mac users who learn unfamiliar visual software by asking spoken questions while working. Use it for contextual guidance where screenshots and pointing reduce ambiguity, especially across multi-monitor workflows. Treat this repository as a hackable reference implementation, not the current product line. It does not replace general automation, keyboard control, or a self-contained offline assistant.
Technical notes+
The native application is primarily Swift and SwiftUI. leanring-buddy/leanring_buddyApp.swift creates a menu-bar-only lifecycle and registers a login item. leanring-buddy/CompanionManager.swift coordinates permissions, push-to-talk state, conversation history, Claude requests, speech playback, and overlay visibility. leanring-buddy/ClaudeAPI.swift posts base64 images and parses streaming SSE text deltas, while leanring-buddy/ElevenLabsTTSClient.swift requests MPEG audio through the proxy and plays it with AVAudioPlayer. leanring-buddy/OverlayWindow.swift implements click-through, per-screen overlays. leanring-buddy/ClickyAnalytics.swift sends transcripts, responses, permission events, and errors to PostHog. README.md documents the Cloudflare deployment and Xcode setup.
Observed
- License
- MIT
- Primary language
- Swift
- Platform support
- macOS 14.2 or newer
- Application interface
- Menu bar app with push-to-talk voice interaction and transparent screen overlays
- Installation surface
- Build the native app with Xcode 15 or newer and install the proxy dependencies with npm
- Service architecture
- Cloudflare Worker proxy for Anthropic, AssemblyAI, and ElevenLabs
- Required permissions
- Microphone, Accessibility, Screen Recording, and Screen Content
Read from README.md, leanring-buddy/ClaudeAPI.swift, leanring-buddy/OpenAIAPI.swift, leanring-buddy/DesignSystem.swift, leanring-buddy/OverlayWindow.swift, leanring-buddy/ClickyAnalytics.swift, leanring-buddy/CompanionManager.swift, leanring-buddy/leanring_buddyApp.swift, leanring-buddy/CompanionPanelView.swift, leanring-buddy/ElevenLabsTTSClient.swift, leanring-buddy/MenuBarPanelManager.swift, leanring-buddy/BuddyDictationManager.swift.
What it can do
Provide real-time screen guidance
Current screen content and user activity → Spoken guidance and instructions
Point to UI elements
Screen elements and user context → Visual indicators highlighting specific interface components
Monitor cursor position
Mouse cursor location and movement → Contextual positioning next to cursor
Analyze screen content
Current display and application state → Understanding of what user is viewing and doing
Deliver audio instructions
Teaching content and context → Spoken explanations and guidance
Provide contextual tutoring
User's current task and screen state → Personalized teaching assistance
Tags
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.