rezayaghobi/agents
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added public chat bale fetch agent

Browse Source
This commit is contained in:
Reza Yaghobi
2026-08-30 22:30:31 +03:30
parent b307977639
commit b52d24daad
3 changed files with 442 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

174
n8n-bale-publicchat-file-fetch/BALE-METRO-FILE-AGENT-DOCS.md Normal file
View File

@@ -0,0 +1,174 @@
# Bale Metro File Agent — Technical Documentation
**What this is:** an n8n workflow that takes a natural-language chat request, figures out which file the user wants, and pulls it out of a *public* Bale bot's conversation history — something the official Bale Bot API cannot do, because a bot token can never see or act inside a chat it doesn't own.
**Status:** working proof of concept. Hardcoded, single-target, built to demonstrate a mechanism — not a production service. Every hardcoded value and known limitation is called out explicitly in this doc so nothing is overstated.
---
## 1. The problem this solves
The original brief: *"you can build this for your own channel, where your bot is admin — but what about a public bot or channel your bot can't be admin of?"*
The Bale **Bot API** (`getUpdates`, `sendMessage`, `getFile`, etc. — the same shape as Telegram's Bot API) only works within chats the bot account is a member/admin of, and only sees activity from the moment it joined onward. There is no bot-token mechanism to:
- open an arbitrary public bot's chat,
- press its buttons,
- or read its message history.
That's a hard protocol boundary, not a missing feature to configure around. Bots generally cannot message other bots or act as a user inside someone else's conversation.
## 2. The solution: client auth instead of bot auth
[`aiobale`](https://github.com/) is an unofficial, community-maintained Python library that implements Bale's actual client protocol — the same websocket/RPC protocol the official Bale app and web client use — rather than the smaller public Bot API. Its `Client` class logs in as a **real phone-number account** (SMS verification code, exactly like opening Bale on a new device), not a bot token.
Once authenticated, that account can do anything a human tapping around the app could: open any public bot's chat, press its buttons, read full history, download files — because from Bale's perspective it *is* a person using the app, not a bot.
This is the core trade-off to be upfront about: **it isn't the "optimal" or intended integration pattern** (it's a real user session, not a first-class bot integration), but it is the only mechanism that can act inside a chat the operator doesn't control.
## 3. Two approaches considered
### Route 1 — press the button, wait for the reply
Send the exact button-label text (e.g. `نقشه خطوط`) into the bot's chat and wait for its next message. Simple and reliable, but it can only ever fetch **one hardcoded thing** — there's no way to decide *which* button to press from a natural-language request without a separate keyword→button mapping table.
Implemented in `fetch_metro_file.py` (kept in the repo as the minimal working example — this is what proved the mechanism worked at all, before the "decide what to fetch" layer was added).
### Route 2 — search history instead of sending a new command *(the one actually used)*
Since the client account is authenticated as a real user, it can call `client.iter_messages()` to read the **entire history** of that bot chat — every file it has ever sent, across every button ever pressed, each with its original caption. Instead of guessing which button maps to a request, the script searches what's already there for a filename/caption/text match.
This is the meaningfully better answer to "can it decide": it doesn't need a hardcoded button map, and it generalizes to anything the bot has ever sent, not just one fixed target.
Implemented in `fetch_metro_smart.py` — this is the script the live n8n workflow calls.
**Honest limitation:** Route 2 can only find files the bot has *already sent at least once*. If nothing matching the keyword has ever appeared in that chat, the search correctly returns "no match" — it does not fall back to pressing a button to fetch something new. Combining both routes (search first, press a button as fallback) is a natural next step, not yet built.
## 4. Architecture
```
n8n (Docker, Linux host) Bale servers
┌─────────────────────────────┐
│ When chat message received │ user types a request in the n8n chat panel
└──────────────┬──────────────┘
▼
┌─────────────────────────────┐
│ Parse request (LLM) │ HTTP POST to BazaarLink's chat-completions
│ │ endpoint. Forces a single-word JSON reply,
│ │ and explicitly instructs the model to
│ │ translate the keyword into Persian, since
│ │ the bot's captions are Persian regardless
│ │ of what language the user typed in.
└──────────────┬──────────────┘
▼
┌─────────────────────────────┐
│ Extract keyword │ Code node: strips markdown fences, parses
│ │ the model's JSON reply into {keyword}.
└──────────────┬──────────────┘
▼
┌─────────────────────────────┐
│ Run client-auth fetch │──────▶ fetch_metro_smart.py (Python, runs
│ (Execute Command node) │ INSIDE the n8n container)
└──────────────┬──────────────┘ │
│ │ aiobale Client, logged in as
│ │ a real Bale account (session
│ │ reused from an earlier SMS
│ │ login — no re-auth per run)
│ ▼
│ iter_messages() over the metro bot's
│ chat history → match filename/caption/
│ text against the keyword → download
│ any match to /mnt/storage/bale-downloads
▼
┌─────────────────────────────┐
│ Parse results │ Code node: reads the script's stdout
│ │ ({"files": [...], "matched": bool}) back
│ │ into a normal n8n item.
└──────────────┬──────────────┘
▼
┌─────────────────────────────┐
│ Save to disk │ Read/Write File node in "read" mode.
│ (visual only) │ The file is ALREADY on disk (the Python
│ │ script wrote it) — this node just reads
│ │ it back so n8n's UI shows the binary
│ │ preview + Download button.
└──────────────────────────────┘
```
**Key point for explaining this out loud:** n8n itself never touches Bale's protocol or the file bytes in transit. It shells out to a Python process that does all of that, and only reads back a small JSON string (a list of file paths) over stdout. The **Save to disk** node's job is purely cosmetic — showing the file nicely in n8n's UI — not actually saving anything.
## 5. Why n8n + Python, not n8n alone
n8n's Code nodes run JavaScript, and there is no JavaScript library for Bale's client protocol. `aiobale` is Python-only, and its `Client` requires a persistent async connection with a completed SMS login — not something that fits inside a stateless per-execution node. So the design bridges the two: n8n orchestrates the natural-language layer (LLM parsing, decision routing, presenting output), and a standalone Python script does the one thing only it can do (speak to Bale as a logged-in user).
## 6. Infrastructure notes (the parts that broke, and why)
These are worth knowing even though they're now resolved — they're the kind of thing an interviewer might ask about directly.
### n8n's Execute Command node is disabled by default (v2+)
Newer n8n images ship with `NODES_EXCLUDE` blocking `Execute Command` for security reasons. Fixed by setting `NODES_EXCLUDE: "[]"` in the container's environment.
### n8n's v2 image dropped `apk`
The official `n8nio/n8n` image is Alpine-based but no longer ships the `apk` package manager itself, so `RUN apk add python3` fails with `apk: not found`. Worked around by copying the `apk` binary and its support files in from a plain `alpine:latest` image in a multi-stage `Dockerfile`, then installing Python 3 and `aiobale-py` into a venv on top of the official n8n image.
### `aiobale` history-scanning bug on large verified bots
`iter_messages()` on the metro bot's chat (596K subscribers) threw a `pydantic.ValidationError`: certain historical messages carry a dict payload (almost certainly a view/seen-tracking structure) in the exact field `aiobale` expects to be plain text (`TextMessage.value: str`). This only affects **history scans**, not live replies — Route 1 never hit it because it never calls `load_history`.
**Fix applied** (top of `fetch_metro_smart.py`): loosen that one field's type at runtime and force pydantic to rebuild its validation schema:
```python
import aiobale.types.message_content as _mc
_mc.TextMessage.model_fields["value"].annotation = Any
_mc.TextMessage.model_rebuild(force=True)
```
Downstream code then defensively checks `isinstance(msg_text, str)` before using it, since the field can now legitimately hold a dict instead of text.
**Scope of the fix, stated honestly:** this patches the *specific* field/shape that actually broke, based on a real traceback — it is not a guarantee that every possible message shape in Bale's protocol is now safe. A different historical message with a different unexpected shape could still throw a fresh, different validation error. This is a known category of risk with unofficial, reverse-engineered protocol libraries.
## 7. File inventory
| File | Purpose |
|---|---|
| `fetch_metro_file.py` | Route 1 (button press + wait) — the first working proof that client auth could talk to a public bot at all. |
| `fetch_metro_smart.py` | Route 2 (history search) — what the live n8n workflow actually calls. Includes the `aiobale` monkeypatch and result deduplication. |
| `fetch_by_description.py` | Earlier variant built for scanning *your own* channel's history (not the metro bot) — kept for reference, not part of the current demo. |
| `bale-metro-file-agent-clientauth.json` | The n8n workflow: chat trigger → LLM keyword parse → Execute Command → parse results → visual "Save to disk". |
| `Dockerfile` | Adds Python 3 + `aiobale-py` on top of the official `n8nio/n8n` image, working around the missing-`apk` issue. |
| `docker-compose.yaml` | Original compose file with two changes: `image:` → `build: .`, and `NODES_EXCLUDE: "[]"` added. |
| `metro_session.bale` | The authenticated session file (JWT + auth metadata) from the one-time phone/SMS login. Lives at `/mnt/storage/bale-downloads/` so it survives container rebuilds via the volume mount. |
## 8. Hardcoded values (everything that would need to change for a different target)
This was built deliberately hardcoded end-to-end for demo purposes — no credential management, no multi-target config. To point this at a different bot/channel, these are the values to change:
- `TARGET_CHAT_ID` (in `fetch_metro_smart.py`) — the numeric `uid` from that chat's `web.bale.ai/chat?uid=...` URL.
- `TARGET_CHAT_TYPE` — `ChatType.BOT` for a bot chat, `ChatType.CHANNEL` for a channel, `ChatType.PRIVATE` for a DM.
- `PHONE_NUMBER` — only used as a fallback if the session file is ever missing/expired and a fresh interactive login is triggered.
- The BazaarLink API key in the workflow JSON's **Parse request (LLM)** node.
## 9. Reproduction steps (from a clean checkout)
1. Place `Dockerfile` next to your `docker-compose.yaml`.
2. In `docker-compose.yaml`: change `image: n8nio/n8n:latest` to `build: .`, and add `NODES_EXCLUDE: "[]"` under the `n8n` service's `environment:`.
3. `docker compose up -d --build`
4. One-time only, on a machine with a terminal/TTY (the container itself has none by default for interactive input): run the login flow to produce `metro_session.bale`, then copy that file into `/mnt/storage/bale-downloads/` (the path the scripts and the volume mount both expect).
5. Copy `fetch_metro_smart.py` into `/mnt/storage/bale-downloads/`.
6. Sanity-check in isolation, before involving n8n at all:
```bash
docker exec -it n8n python3 /mnt/storage/bale-downloads/fetch_metro_smart.py خطوط
```
Expect `{"files": ["...tehran-subway1.jpg"], "matched": true}`.
7. Import `bale-metro-file-agent-clientauth.json` into n8n (**Workflows → Import from File**).
8. Open the workflow's chat panel and type a request in any language, e.g. *"get me the metro map."*
## 10. What this honestly proves vs. doesn't
**Proven:**
- Client auth (not bot auth) can access a chat/bot the operator doesn't control.
- A natural-language request can be turned into a targeted search over that chat's real history, in a different language than the underlying content.
- The mechanism survives real infrastructure friction (container security defaults, missing OS package manager, an upstream library bug) — all diagnosed and fixed with evidence (tracebacks), not guesswork.
**Not proven / explicitly out of scope for this demo:**
- Generalization to *any* bot/channel without manually changing the hardcoded target — no dynamic bot discovery.
- Combining Route 1 and Route 2 (falling back to pressing a button when history search comes up empty).
- Production concerns: no credential vaulting, no per-chat request locking for concurrent use, no retry/backoff, no handling for further undiscovered `aiobale` parsing edge cases beyond the one hit here.
- The account used for client auth is a real personal Bale account — rate limits, terms-of-service exposure, and account-level risk apply in a way they wouldn't for an official bot integration. Worth stating plainly if asked: this trades "it works against public chats" for "it's not the sanctioned integration path."

176
n8n-bale-publicchat-file-fetch/json/Bale Metro File Agent (client-auth demo).json Normal file
View File

@@ -0,0 +1,176 @@
{
"name": "Bale Metro File Agent (client-auth demo)",
"nodes": [
{
"parameters": {
"options": {}
},
"id": "6bb427a1-bb35-4742-93e6-ebb320950224",
"name": "When chat message received",
"type": "@n8n/n8n-nodes-langchain.chatTrigger",
"typeVersion": 1.1,
"position": [
3344,
384
],
"webhookId": "fdae82fa-cb2d-4d9a-83ba-4ae9bddd0a6e"
},
{
"parameters": {
"method": "POST",
"url": "https://api.bazaarlink.ai/v1/chat/completions",
"sendHeaders": true,
"headerParameters": {
"parameters": [
{
"name": "Authorization",
"value": "Bearer sk-bl-RszuRzm4SBYwRapwenmPDyHMNZnxVkZ3w8UxtNPArWTFos23"
},
{
"name": "Content-Type",
"value": "application/json"
}
]
},
"sendBody": true,
"specifyBody": "json",
"jsonBody": "={{ JSON.stringify({ model: \"auto:free\", messages: [ { role: \"system\", content: \"You extract a single short search keyword describing the file the user wants. IMPORTANT: the files you are searching are Bale bot messages with Persian (Farsi) captions and filenames -- no matter what language the user writes in, output the keyword translated into a plausible Persian word likely to appear in that caption (e.g. a request for 'the metro map' should become the keyword 'نقشه' or 'خطوط', not 'map'). Reply with ONLY compact JSON like {\\\"keyword\\\":\\\"خطوط\\\"}. No prose, no markdown fences.\" }, { role: \"user\", content: $json.chatInput } ] }) }}",
"options": {}
},
"id": "47b99f54-f43e-4a16-b281-5a20bf8150a5",
"name": "Parse request (LLM)",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.2,
"position": [
3792,
384
]
},
{
"parameters": {
"jsCode": "let raw = $input.first().json.choices[0].message.content.trim();\nraw = raw.replace(/^```json\\s*/i, '').replace(/^```\\s*/i, '').replace(/```\\s*$/i, '');\nconst parsed = JSON.parse(raw);\nreturn [{ json: parsed }];"
},
"id": "1156dfdb-4cae-49fb-92c2-cfaa35ff86e0",
"name": "Extract keyword",
"type": "n8n-nodes-base.code",
"typeVersion": 2,
"position": [
4016,
384
]
},
{
"parameters": {
"command": "=python3 /mnt/storage/bale-downloads/fetch_metro_smart.py \"{{ $json.keyword }}\""
},
"id": "ede1882a-c5df-4074-a1a0-4bfac8dca00e",
"name": "Run client-auth fetch",
"type": "n8n-nodes-base.executeCommand",
"typeVersion": 1,
"position": [
4240,
384
],
"notes": "Runs fetch_metro_smart.py, passing the LLM-parsed keyword through. The script searches this bot chat's full history for a document whose filename/caption matches the keyword -- it does NOT send a new command, it decides from what's already there."
},
{
"parameters": {
"jsCode": "const result = JSON.parse($input.first().json.stdout);\nconst files = result.files || [];\nreturn [{ json: {\n found: files.length,\n files,\n summary: files.length\n ? `Found ${files.length} file(s):\\n` + files.join('\\n')\n : 'No file in this chat\\'s history matched that description.'\n} }];"
},
"id": "18f2ad30-f652-45a1-a825-b57183d498ca",
"name": "Parse results",
"type": "n8n-nodes-base.code",
"typeVersion": 2,
"position": [
4464,
384
]
},
{
"parameters": {
"fileSelector": "={{ $json.files[0] }}",
"options": {}
},
"id": "95ca5a02-c89c-4b90-b476-69f1b3382de9",
"name": "Save to disk",
"type": "n8n-nodes-base.readWriteFile",
"typeVersion": 1,
"position": [
4688,
384
],
"notes": "The file is already on disk (the Python script wrote it). This node just reads it back so n8n shows the binary preview + Download button, same as the original Bot-API demo -- purely visual."
}
],
"pinData": {},
"connections": {
"When chat message received": {
"main": [
[
{
"node": "Parse request (LLM)",
"type": "main",
"index": 0
}
]
]
},
"Parse request (LLM)": {
"main": [
[
{
"node": "Extract keyword",
"type": "main",
"index": 0
}
]
]
},
"Extract keyword": {
"main": [
[
{
"node": "Run client-auth fetch",
"type": "main",
"index": 0
}
]
]
},
"Run client-auth fetch": {
"main": [
[
{
"node": "Parse results",
"type": "main",
"index": 0
}
]
]
},
"Parse results": {
"main": [
[
{
"node": "Save to disk",
"type": "main",
"index": 0
}
]
]
}
},
"active": false,
"settings": {
"executionOrder": "v1",
"binaryMode": "separate",
"availableInMCP": false
},
"versionId": "a1d3946d-dfda-478a-93eb-e11cb88d4722",
"meta": {
"instanceId": "a8618b8efe9a0b8c68300d12dc12f140ace3ebe855b80198b7819ca3e0922fe9"
},
"nodeGroups": [],
"id": "L9iWiHBHGxQZDvGU",
"tags": []
}

92
n8n-bale-publicchat-file-fetch/script/fetch_metro_smart.py Normal file
View File

@@ -0,0 +1,92 @@
"""
fetch_metro_smart.py -- decides which file to grab, instead of always
sending the same button.
Route 2: since we're authenticated as a real user (not a bot token), we can
read the FULL history of this bot chat, not just react to new messages.
Every file the bot has ever sent you (from any button you've pressed before)
is sitting in that history with a caption. So instead of guessing which
button maps to which request, we search what's already there.
Usage (n8n passes the LLM-parsed keyword as the one argument):
python3 fetch_metro_smart.py "خطوط"
Prints one line of JSON to stdout: {"files": [...], "matched": true/false}
"""
import asyncio
import json
import sys
import pathlib
from typing import Any
from aiobale import Client, Dispatcher
from aiobale.enums import ChatType
# ---- upstream library workaround ---------------------------------------
# aiobale hard-types TextMessage.value as `str`, but Bale's servers attach a
# view/seen-tracking payload (a dict) to some messages from large verified
# bots -- history scans on those chats crash with a pydantic ValidationError
# before we ever see the message. This loosens that one field so parsing
# succeeds; matches() below coerces to str defensively since the value can
# now legitimately be a dict instead of text.
import aiobale.types.message_content as _mc
_mc.TextMessage.model_fields["value"].annotation = Any
_mc.TextMessage.model_rebuild(force=True)
# -------------------------------------------------------------------------
# ---- hardcoded demo config --------------------------------------------
SESSION_FILE = "/mnt/storage/bale-downloads/metro_session.bale"
PHONE_NUMBER = 989381393442
TARGET_CHAT_ID = 1381242617 # متروی تهران و حومه
TARGET_CHAT_TYPE = ChatType.BOT
DOWNLOAD_DIR = pathlib.Path("/mnt/storage/bale-downloads")
MAX_MESSAGES = 200 # how far back into this chat's history to scan
# -------------------------------------------------------------------------
DOWNLOAD_DIR.mkdir(parents=True, exist_ok=True)
dp = Dispatcher()
client = Client(dp, session_file=SESSION_FILE, phone_number=PHONE_NUMBER)
def matches(doc, msg, keyword: str) -> bool:
keyword = (keyword or "").lower().strip()
if not keyword:
return False # no keyword = match nothing (avoid grabbing everything by accident)
caption = doc.caption.content if doc.caption else ""
msg_text = msg.text
if not isinstance(msg_text, str):
msg_text = "" # some messages carry a non-text payload in this field -- see workaround above
haystack = " ".join(filter(None, [doc.name or "", caption or "", msg_text])).lower()
return keyword in haystack
async def main(keyword: str):
await client.start(run_in_background=True)
await asyncio.sleep(2)
saved = []
try:
async for msg in client.iter_messages(
chat_id=TARGET_CHAT_ID, chat_type=TARGET_CHAT_TYPE, limit=MAX_MESSAGES
):
doc = msg.document
if not doc:
continue
if matches(doc, msg, keyword):
dest = DOWNLOAD_DIR / (doc.name or f"{msg.message_id}.bin")
if str(dest) in saved:
continue # already matched this exact file earlier in history
await msg.download(destination=dest)
saved.append(str(dest))
finally:
await client.stop()
print(json.dumps({"files": saved, "matched": len(saved) > 0}))
if __name__ == "__main__":
kw = sys.argv[1] if len(sys.argv) > 1 else ""
asyncio.run(main(kw))