fix(hs): probe onion reachability instead of waiting on Bootstrapping
All checks were successful
ci / test (push) Successful in 2m59s
All checks were successful
ci / test (push) Successful in 2m59s
Arti combined status stays Bootstrapping through a 5 min HsDir upload round. Treat a successful connect as published, floor cbtinitialtimeout with cbtmintimeout at 20s, and skip preemptive 80/443 circuits so IPT and HsDir builds are not starved. PUBLISH_WAIT stays 360s fail-closed.
This commit is contained in:
parent
e63203ff56
commit
e902a9041d
5 changed files with 130 additions and 24 deletions
|
|
@ -106,7 +106,7 @@ onionwire
|
||||||
|
|
||||||
or `cargo run --release`.
|
or `cargo run --release`.
|
||||||
|
|
||||||
On start you should see `onionwire: bootstrapping Arti…` on stderr. First bootstrap can take a minute. Data lives in `ONIONWIRE_HOME` if set, otherwise `~/.local/share/onionwire/` (`onionwire.db` + Arti state, mode 0700). First open creates an ed25519 identity key. That key **is** you.
|
On start you should see `onionwire: bootstrapping Arti…` on stderr. Directory bootstrap is usually under a minute; the onion is ready once a probe connect works (combined Arti status may still say Bootstrapping). Fail closed at 360s. Data lives in `ONIONWIRE_HOME` if set, otherwise `~/.local/share/onionwire/` (`onionwire.db` + Arti state, mode 0700). First open creates an ed25519 identity key. That key **is** you.
|
||||||
|
|
||||||
## Friends are keys
|
## Friends are keys
|
||||||
|
|
||||||
|
|
@ -255,7 +255,7 @@ Do not run `cargo publish`; `publish = false`. CI needs the repo secret
|
||||||
|---|---|
|
|---|---|
|
||||||
| `GLIBC_… not found` | Binary is newer than your libc. Build from source (Option B). |
|
| `GLIBC_… not found` | Binary is newer than your libc. Build from source (Option B). |
|
||||||
| `sha256sum: FAILED` | Re-download both the binary and `.sha256`; run the check in the same directory. |
|
| `sha256sum: FAILED` | Re-download both the binary and `.sha256`; run the check in the same directory. |
|
||||||
| Hang on `bootstrapping Arti…` / `hs status: Bootstrapping` | Outbound network required. Client bootstrap is minutes; HsDir descriptor upload can stay Bootstrapping up to ~6 min before `DegradedReachable`. Fail closed after that — no C-tor fallback. |
|
| Hang on `bootstrapping Arti…` / `hs status: Bootstrapping` | Outbound network required. Combined Arti status can stay Bootstrapping through a 5 min HsDir upload round; OnionWire probes the onion and proceeds once a connect works. Still fail closed after 360s if neither a probe nor `DegradedReachable`/`Running` — no C-tor fallback. |
|
||||||
| `onionwire: unknown argument` | No subcommands. Flags are `--version` / `--help` only, then the TUI. |
|
| `onionwire: unknown argument` | No subcommands. Flags are `--version` / `--help` only, then the TUI. |
|
||||||
| Blank / broken TUI | Run in a real terminal emulator, not `nohup` / systemd without a TTY. |
|
| Blank / broken TUI | Run in a real terminal emulator, not `nohup` / systemd without a TTY. |
|
||||||
| Two chats, same laptop | Separate `ONIONWIRE_HOME` per process. |
|
| Two chats, same laptop | Separate `ONIONWIRE_HOME` per process. |
|
||||||
|
|
|
||||||
75
src/hs.rs
75
src/hs.rs
|
|
@ -15,7 +15,16 @@ pub const HS_PORT: u16 = 80;
|
||||||
/// HsDir descriptor upload often stays Bootstrapping past 3 minutes.
|
/// HsDir descriptor upload often stays Bootstrapping past 3 minutes.
|
||||||
pub const PUBLISH_WAIT: Duration = Duration::from_secs(360);
|
pub const PUBLISH_WAIT: Duration = Duration::from_secs(360);
|
||||||
/// Consensus default `cbtmintimeout` is 10ms; learned CBT can drop to ~1s.
|
/// Consensus default `cbtmintimeout` is 10ms; learned CBT can drop to ~1s.
|
||||||
pub const CBT_MIN_TIMEOUT_MS: i32 = 10_000;
|
/// 4-hop vanguard HS circuits need more than 10s on a slow net.
|
||||||
|
pub const CBT_MIN_TIMEOUT_MS: i32 = 20_000;
|
||||||
|
/// Consensus `cbtinitialtimeout` can be ~2s; match the min floor.
|
||||||
|
pub const CBT_INITIAL_TIMEOUT_MS: i32 = CBT_MIN_TIMEOUT_MS;
|
||||||
|
/// OnionWire never uses exit ports. Default 80/443 preemptive circuits
|
||||||
|
/// compete with IPT + HsDir builds during publish.
|
||||||
|
pub const PREEMPTIVE_PREDICTED_PORTS: &[u16] = &[];
|
||||||
|
|
||||||
|
const PROBE_EVERY: Duration = Duration::from_secs(15);
|
||||||
|
const PROBE_TIMEOUT: Duration = Duration::from_secs(12);
|
||||||
|
|
||||||
pub type Client = Arc<TorClient<PreferredRuntime>>;
|
pub type Client = Arc<TorClient<PreferredRuntime>>;
|
||||||
|
|
||||||
|
|
@ -27,6 +36,12 @@ pub fn client_config(state_dir: &std::path::Path, cache_dir: &std::path::Path) -
|
||||||
builder
|
builder
|
||||||
.override_net_params()
|
.override_net_params()
|
||||||
.insert("cbtmintimeout".to_string(), CBT_MIN_TIMEOUT_MS);
|
.insert("cbtmintimeout".to_string(), CBT_MIN_TIMEOUT_MS);
|
||||||
|
builder
|
||||||
|
.override_net_params()
|
||||||
|
.insert("cbtinitialtimeout".to_string(), CBT_INITIAL_TIMEOUT_MS);
|
||||||
|
builder
|
||||||
|
.preemptive_circuits()
|
||||||
|
.set_initial_predicted_ports(PREEMPTIVE_PREDICTED_PORTS.to_vec());
|
||||||
builder.build().expect("TorClientConfig")
|
builder.build().expect("TorClientConfig")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -55,29 +70,61 @@ pub fn onion_string(svc: &RunningOnionService) -> Result<String, String> {
|
||||||
Ok(id.display_unredacted().to_string())
|
Ok(id.display_unredacted().to_string())
|
||||||
}
|
}
|
||||||
|
|
||||||
pub async fn wait_until_published(svc: &RunningOnionService, label: &str) -> Result<(), String> {
|
/// Combined Arti status stays Bootstrapping through a 5 min HsDir upload
|
||||||
|
/// round. A successful connect means some HsDir already has the descriptor.
|
||||||
|
pub fn hs_is_ready(state: State, probe_ok: bool) -> bool {
|
||||||
|
match state {
|
||||||
|
State::Broken | State::Shutdown => false,
|
||||||
|
State::Running | State::DegradedReachable => true,
|
||||||
|
_ => probe_ok,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub async fn wait_until_published(
|
||||||
|
client: &Client,
|
||||||
|
svc: &RunningOnionService,
|
||||||
|
onion: &str,
|
||||||
|
label: &str,
|
||||||
|
) -> Result<(), String> {
|
||||||
let deadline = Instant::now() + PUBLISH_WAIT;
|
let deadline = Instant::now() + PUBLISH_WAIT;
|
||||||
let mut events = svc.status_events();
|
let mut events = svc.status_events();
|
||||||
|
let mut next_probe = Instant::now() + PROBE_EVERY;
|
||||||
loop {
|
loop {
|
||||||
let st = svc.status();
|
let st = svc.status();
|
||||||
eprintln!("{label} hs status: {:?}", st.state());
|
let state = st.state();
|
||||||
match st.state() {
|
eprintln!("{label} hs status: {state:?}");
|
||||||
State::Running | State::DegradedReachable => return Ok(()),
|
if hs_is_ready(state, false) {
|
||||||
State::Broken => {
|
return Ok(());
|
||||||
return Err(format!(
|
}
|
||||||
"{label}: onion service broken: {:?}",
|
if matches!(state, State::Broken) {
|
||||||
st.current_problem()
|
return Err(format!(
|
||||||
));
|
"{label}: onion service broken: {:?}",
|
||||||
}
|
st.current_problem()
|
||||||
_ => {}
|
));
|
||||||
}
|
}
|
||||||
if Instant::now() >= deadline {
|
if Instant::now() >= deadline {
|
||||||
return Err(format!(
|
return Err(format!(
|
||||||
"{label}: onion service did not publish within {}s: {:?}",
|
"{label}: onion service did not publish within {}s: {state:?}",
|
||||||
PUBLISH_WAIT.as_secs(),
|
PUBLISH_WAIT.as_secs(),
|
||||||
st.state()
|
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
|
let now = Instant::now();
|
||||||
|
if now >= next_probe {
|
||||||
|
eprintln!("{label} probing reachability");
|
||||||
|
let probe_ok = tokio::time::timeout(PROBE_TIMEOUT, client.connect((onion, HS_PORT)))
|
||||||
|
.await
|
||||||
|
.ok()
|
||||||
|
.and_then(Result::ok)
|
||||||
|
.is_some();
|
||||||
|
if probe_ok {
|
||||||
|
eprintln!("{label} hs reachable while status {state:?}");
|
||||||
|
}
|
||||||
|
if hs_is_ready(state, probe_ok) {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
next_probe = Instant::now() + PROBE_EVERY;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
tokio::select! {
|
tokio::select! {
|
||||||
_ = events.next() => {}
|
_ = events.next() => {}
|
||||||
_ = tokio::time::sleep(Duration::from_secs(2)) => {}
|
_ = tokio::time::sleep(Duration::from_secs(2)) => {}
|
||||||
|
|
|
||||||
24
src/node.rs
24
src/node.rs
|
|
@ -63,20 +63,29 @@ impl Node {
|
||||||
.ok_or_else(|| "onion service disabled in config — fail closed".to_string())?;
|
.ok_or_else(|| "onion service disabled in config — fail closed".to_string())?;
|
||||||
let (svc, rend) = launched;
|
let (svc, rend) = launched;
|
||||||
let onion = hs::onion_string(&svc)?;
|
let onion = hs::onion_string(&svc)?;
|
||||||
hs::wait_until_published(&svc, &onion).await?;
|
|
||||||
store.set_onion(&onion).map_err(|e| e.to_string())?;
|
|
||||||
let node = Arc::new(Self {
|
let node = Arc::new(Self {
|
||||||
home,
|
home,
|
||||||
store: Mutex::new(store),
|
store: Mutex::new(store),
|
||||||
client,
|
client,
|
||||||
hs: Mutex::new(None),
|
hs: Mutex::new(None),
|
||||||
onion: Mutex::new(onion),
|
onion: Mutex::new(onion.clone()),
|
||||||
keys: Mutex::new(keys),
|
keys: Mutex::new(keys),
|
||||||
wallet: Wallet::from_env(),
|
wallet: Wallet::from_env(),
|
||||||
incoming_limit: Mutex::new(TokenBucket::default()),
|
incoming_limit: Mutex::new(TokenBucket::default()),
|
||||||
});
|
});
|
||||||
|
// Accept rens before waiting so a reachability probe can succeed
|
||||||
|
// while combined status is still Bootstrapping.
|
||||||
let rend = spawn_rend(Arc::clone(&node), rend);
|
let rend = spawn_rend(Arc::clone(&node), rend);
|
||||||
*node.hs.lock().map_err(|e| e.to_string())? = Some(HsHandle { _svc: svc, rend });
|
*node.hs.lock().map_err(|e| e.to_string())? = Some(HsHandle {
|
||||||
|
_svc: Arc::clone(&svc),
|
||||||
|
rend,
|
||||||
|
});
|
||||||
|
hs::wait_until_published(&node.client, &svc, &onion, &onion).await?;
|
||||||
|
node.store
|
||||||
|
.lock()
|
||||||
|
.map_err(|e| e.to_string())?
|
||||||
|
.set_onion(&onion)
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
Ok(node)
|
Ok(node)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -400,9 +409,12 @@ impl Node {
|
||||||
let onion = hs::onion_string(&svc)?;
|
let onion = hs::onion_string(&svc)?;
|
||||||
// ponytail: hard-cut old HS before waiting; keeping both stalled ow1 at Bootstrapping.
|
// ponytail: hard-cut old HS before waiting; keeping both stalled ow1 at Bootstrapping.
|
||||||
*self.hs.lock().map_err(|e| e.to_string())? = None;
|
*self.hs.lock().map_err(|e| e.to_string())? = None;
|
||||||
hs::wait_until_published(&svc, &onion).await?;
|
|
||||||
let rend = spawn_rend(Arc::clone(self), rend);
|
let rend = spawn_rend(Arc::clone(self), rend);
|
||||||
*self.hs.lock().map_err(|e| e.to_string())? = Some(HsHandle { _svc: svc, rend });
|
*self.hs.lock().map_err(|e| e.to_string())? = Some(HsHandle {
|
||||||
|
_svc: Arc::clone(&svc),
|
||||||
|
rend,
|
||||||
|
});
|
||||||
|
hs::wait_until_published(&self.client, &svc, &onion, &onion).await?;
|
||||||
{
|
{
|
||||||
let store = self.store.lock().map_err(|e| e.to_string())?;
|
let store = self.store.lock().map_err(|e| e.to_string())?;
|
||||||
store.set_onion(&onion).map_err(|e| e.to_string())?;
|
store.set_onion(&onion).map_err(|e| e.to_string())?;
|
||||||
|
|
|
||||||
47
tests/hs.rs
47
tests/hs.rs
|
|
@ -3,6 +3,7 @@
|
||||||
use std::time::Duration;
|
use std::time::Duration;
|
||||||
|
|
||||||
use onionwire::hs;
|
use onionwire::hs;
|
||||||
|
use tor_hsservice::status::State;
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn publish_wait_covers_hsdir_retries() {
|
fn publish_wait_covers_hsdir_retries() {
|
||||||
|
|
@ -20,3 +21,49 @@ fn cbt_min_timeout_floor_is_at_least_10s() {
|
||||||
let floor_ms = hs::CBT_MIN_TIMEOUT_MS;
|
let floor_ms = hs::CBT_MIN_TIMEOUT_MS;
|
||||||
assert!(floor_ms >= 10_000, "learned CBT ~1s kills HsDir circuits");
|
assert!(floor_ms >= 10_000, "learned CBT ~1s kills HsDir circuits");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn cbt_initial_timeout_matches_the_floor() {
|
||||||
|
// Consensus cbtinitialtimeout can be ~2s; 4-hop vanguard HS circuits
|
||||||
|
// need the same floor as cbtmintimeout before the estimator has samples.
|
||||||
|
let initial = hs::CBT_INITIAL_TIMEOUT_MS;
|
||||||
|
let floor = hs::CBT_MIN_TIMEOUT_MS;
|
||||||
|
assert!(initial >= floor, "initial CBT below min floor");
|
||||||
|
assert!(initial >= 10_000, "initial CBT too low for HsDir circuits");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn no_preemptive_exit_ports() {
|
||||||
|
// Default predicted 80/443 circuits compete with IPT + HsDir builds.
|
||||||
|
// OnionWire never exits; keep the predicted list empty.
|
||||||
|
assert!(
|
||||||
|
hs::PREEMPTIVE_PREDICTED_PORTS.is_empty(),
|
||||||
|
"preemptive exit circuits starve HS publish"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn arti_status_running_is_ready_without_probe() {
|
||||||
|
assert!(hs::hs_is_ready(State::Running, false));
|
||||||
|
assert!(hs::hs_is_ready(State::DegradedReachable, false));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn bootstrapping_is_not_ready_until_a_probe_connects() {
|
||||||
|
// Combined status stays Bootstrapping through Arti's 5 min HsDir upload
|
||||||
|
// round even after some descriptors are already fetchable.
|
||||||
|
assert!(!hs::hs_is_ready(State::Bootstrapping, false));
|
||||||
|
assert!(hs::hs_is_ready(State::Bootstrapping, true));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn degraded_unreachable_is_ready_only_if_probe_connects() {
|
||||||
|
assert!(!hs::hs_is_ready(State::DegradedUnreachable, false));
|
||||||
|
assert!(hs::hs_is_ready(State::DegradedUnreachable, true));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn broken_or_shutdown_never_ready() {
|
||||||
|
assert!(!hs::hs_is_ready(State::Broken, true));
|
||||||
|
assert!(!hs::hs_is_ready(State::Shutdown, true));
|
||||||
|
}
|
||||||
|
|
|
||||||
|
|
@ -124,10 +124,10 @@ async fn two_node_byte_pipe_restart_and_dormant() {
|
||||||
eprintln!("bob onion={bob_onion}");
|
eprintln!("bob onion={bob_onion}");
|
||||||
assert_ne!(alice_onion, bob_onion, "separate HS identities");
|
assert_ne!(alice_onion, bob_onion, "separate HS identities");
|
||||||
|
|
||||||
hs::wait_until_published(&alice_svc, "alice")
|
hs::wait_until_published(&alice, &alice_svc, &alice_onion, "alice")
|
||||||
.await
|
.await
|
||||||
.expect("alice publish");
|
.expect("alice publish");
|
||||||
hs::wait_until_published(&bob_svc, "bob")
|
hs::wait_until_published(&bob, &bob_svc, &bob_onion, "bob")
|
||||||
.await
|
.await
|
||||||
.expect("bob publish");
|
.expect("bob publish");
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue