[pmg-devel] [PATCH v2 pmg-log-tracker] rewrite in rust
Mira Limbeck
m.limbeck at proxmox.com
Tue Nov 5 16:58:20 CET 2019
pmg-log-tracker has been rewritten in Rust. Functionality is the same.
Output sometimes has a different order than the pmg-log-tracker in C.
This only happens when the time of the entries match.
There's one change regarding the interface. In addition to the short
versions of arguments also long versions exist.
The implementation uses Rc<> and RefCell<> to make holding mutable
cross-references possible, without having to change the original logic
completely. This allowed for easier translation from C to Rust.
The file debian/cargo-checksum.json is required by dh-cargo, otherwise
it won't compile. The cargo-checksum.json should contain the upstream
.crate file which does not exist in this case, so we just create an
empty one with the required keys. (see 'Library package structure' in
https://wiki.debian.org/Teams/RustPackaging/Policy)
The change to the minimum version of debhelper required was done
according to other rust packages (rust-clap, rust-bindgen, rust-ripgrep).
Removes src/pmg-log-tracker.c and src/Makefile as those are no longer
required.
Signed-off-by: Mira Limbeck <m.limbeck at proxmox.com>
---
v2:
- fixed an issue where the incorrect number of files got returned by
Parser::count_relevant_files()
- remove src/pmg-log-tracker.c and src/Makefile
- added simple benchmarks
Regarding the version, how do you want to handle it? Keep the Cargo.toml
version in sync with the package version (major, minor)?
This already contains the changes @Wolfgang recommended, simplifying a
lot of code, especially the printing. Also got rid of one RefCell and
one Cell usage, as we can now pass the Parser as mutable reference to
both print() functions.
Some simple benchmarks: (32 syslog files (syslog to syslog.31.gz))
Rust: (median of 5 runs + 1 for cache)
sudo pmg-log-tracker -s "2018-01-01 00:00:00" -vv > /dev/null 11.68s user 0.74s system 99% cpu 12.421 total
sudo pmg-log-tracker -s "2018-01-01 00:00:00" -v > /dev/null 9.89s user 0.64s system 99% cpu 10.531 total
C: (same as for the rust version)
sudo pmg-log-tracker -s "2018-01-01 00:00:00" -vv > /dev/null 11.83s user 0.32s system 99% cpu 12.147 total
sudo pmg-log-tracker -s "2018-01-01 00:00:00" -v > /dev/null 10.58s user 0.24s system 99% cpu 10.821 total
The -v version is used by the GUI.
Cargo.toml | 12 +
Makefile | 4 +-
debian/cargo-checksum.json | 1 +
debian/control | 17 +-
debian/rules | 2 +-
src/Makefile | 20 -
src/main.rs | 1969 +++++++++++++++++++++++++++
src/pmg-log-tracker.c | 2580 ------------------------------------
8 files changed, 2000 insertions(+), 2605 deletions(-)
create mode 100644 Cargo.toml
create mode 100644 debian/cargo-checksum.json
delete mode 100644 src/Makefile
create mode 100644 src/main.rs
delete mode 100644 src/pmg-log-tracker.c
diff --git a/Cargo.toml b/Cargo.toml
new file mode 100644
index 0000000..4cf75f1
--- /dev/null
+++ b/Cargo.toml
@@ -0,0 +1,12 @@
+[package]
+name = "pmg-log-tracker"
+version = "2.1.0"
+authors = ["Mira Limbeck <m.limbeck at proxmox.com>", "Dietmar Maurer <dietmar at proxmox.com>"]
+edition = "2018"
+
+[dependencies]
+clap = "2.32.0"
+failure = "0.1.5"
+flate2 = "1.0.6"
+libc = "0.2.48"
+time = "0.1.42"
diff --git a/Makefile b/Makefile
index 1e344dd..b50fe02 100644
--- a/Makefile
+++ b/Makefile
@@ -14,7 +14,9 @@ all: ${DEB}
.PHONY: ${BUILDDIR}
${BUILDDIR}: src
rm -rf ${BUILDDIR} ${BUILDDIR}.tmp
- cp -a src ${BUILDDIR}.tmp
+ mkdir ${BUILDDIR}.tmp
+ cp -a src ${BUILDDIR}.tmp/src
+ cp Cargo.toml ${BUILDDIR}.tmp/
cp -a debian ${BUILDDIR}.tmp/debian
echo "git clone git://git.proxmox.com/git/pmg-log-tracker.git\\ngit checkout ${GITVERSION}" > ${BUILDDIR}.tmp/debian/SOURCE
mv ${BUILDDIR}.tmp ${BUILDDIR}
diff --git a/debian/cargo-checksum.json b/debian/cargo-checksum.json
new file mode 100644
index 0000000..ce2579b
--- /dev/null
+++ b/debian/cargo-checksum.json
@@ -0,0 +1 @@
+{"package":"","files":{}}
diff --git a/debian/control b/debian/control
index 26210a3..e02c296 100644
--- a/debian/control
+++ b/debian/control
@@ -2,13 +2,24 @@ Source: pmg-log-tracker
Section: admin
Priority: optional
Maintainer: Proxmox Support Team <support at proxmox.com>
-Build-Depends: debhelper (>= 10~),
- libglib2.0-dev (>= 2.42.1)
+Build-Depends: cargo:native,
+ debhelper (>= 11),
+ dh-cargo (>= 10),
+ librust-clap-dev,
+ librust-failure-dev,
+ librust-flate2-dev,
+ librust-libc-dev,
+ librust-time-dev,
+ libstd-rust-dev,
+ rustc:native
Standards-Version: 3.9.8
Homepage: http://www.proxmox.com
Package: pmg-log-tracker
Architecture: any
-Depends: ${shlibs:Depends}, ${misc:Depends}
+Depends: ${misc:Depends},
+ ${shlibs:Depends}
+Built-Using: ${cargo:Built-Using}
+XB-X-Cargo-Built-Using: ${cargo:X-Cargo-Built-Using}
Description: Proxmox Mailgateway Log Tracker
Tools to scan mail logs.
diff --git a/debian/rules b/debian/rules
index a7521b6..e3ba5d4 100755
--- a/debian/rules
+++ b/debian/rules
@@ -5,4 +5,4 @@
%:
- dh $@
+ dh $@ --buildsystem cargo
diff --git a/src/Makefile b/src/Makefile
deleted file mode 100644
index 1f48092..0000000
--- a/src/Makefile
+++ /dev/null
@@ -1,20 +0,0 @@
-DESTDIR=
-
-LIBS=$(shell pkg-config --libs glib-2.0) -lz
-CFLAGS=$(shell pkg-config --cflags glib-2.0) -O2 -Wpedantic
-
-all: pmg-log-tracker
-
-pmg-log-tracker: pmg-log-tracker.c
- gcc $< -o $@ ${CFLAGS} ${LIBS}
-
-.PHONY: install
-install: pmg-log-tracker
- install -d ${DESTDIR}/usr/bin
- install -m 0755 pmg-log-tracker ${DESTDIR}/usr/bin/
-
-.PHONY: clean distclean
-distclean: clean
-clean:
- rm -rf pmg-log-tracker
- find . -name '*~' -exec rm {} ';'
diff --git a/src/main.rs b/src/main.rs
new file mode 100644
index 0000000..db9d45c
--- /dev/null
+++ b/src/main.rs
@@ -0,0 +1,1969 @@
+#[macro_use]
+extern crate clap;
+extern crate failure;
+extern crate flate2;
+extern crate libc;
+
+use std::cell::RefCell;
+use std::collections::HashMap;
+use std::ffi::CString;
+use std::rc::Rc;
+
+use std::fs::File;
+use std::io::BufRead;
+use std::io::BufReader;
+use std::io::BufWriter;
+use std::io::Write;
+
+use failure::Error;
+use flate2::read;
+
+use clap::{App, Arg};
+
+/// Parse a QID ([A-Z]+). Returns a tuple of (qid, remaining_text) or None.
+fn parse_qid(data: &[u8], max: usize) -> Option<(&[u8], &[u8])> {
+ let mut bytes = data.iter();
+ let mut count = 0;
+ while count < max {
+ match bytes.next() {
+ Some(c) => {
+ if c.is_ascii_hexdigit() {
+ count += 1;
+ } else {
+ break;
+ }
+ }
+ None => break,
+ }
+ }
+ if count > 1 {
+ return Some((&data[..count], &data[count..]));
+ }
+ None
+}
+
+fn main() -> Result<(), Error> {
+ let matches = App::new(crate_name!())
+ .version(crate_version!())
+ .about(crate_description!())
+ .arg(
+ Arg::with_name("verbose")
+ .short("v")
+ .long("verbose")
+ .help("Verbose output, can be specified multiple times")
+ .multiple(true)
+ .takes_value(false),
+ )
+ .arg(
+ Arg::with_name("inputfile")
+ .short("i")
+ .long("inputfile")
+ .help("Input file to use instead of /var/log/syslog, or '-' for stdin")
+ .value_name("INPUTFILE"),
+ )
+ .arg(
+ Arg::with_name("host")
+ .short("h")
+ .long("host")
+ .help("Hostname or Server IP")
+ .value_name("HOST"),
+ )
+ .arg(
+ Arg::with_name("from")
+ .short("f")
+ .long("from")
+ .help("Mails from SENDER")
+ .value_name("SENDER"),
+ )
+ .arg(
+ Arg::with_name("to")
+ .short("t")
+ .long("to")
+ .help("Mails to RECIPIENT")
+ .value_name("RECIPIENT"),
+ )
+ .arg(
+ Arg::with_name("start")
+ .short("s")
+ .long("starttime")
+ .help("Start time (YYYY-MM-DD HH:MM:SS) or seconds since epoch")
+ .value_name("TIME"),
+ )
+ .arg(
+ Arg::with_name("end")
+ .short("e")
+ .long("endtime")
+ .help("End time (YYYY-MM-DD HH:MM:SS) or seconds since epoch")
+ .value_name("TIME"),
+ )
+ .arg(
+ Arg::with_name("msgid")
+ .short("m")
+ .long("message-id")
+ .help("Message ID (exact match)")
+ .value_name("MSGID"),
+ )
+ .arg(
+ Arg::with_name("qids")
+ .short("q")
+ .long("queue-id")
+ .help("Queue ID (exact match), can be specified multiple times")
+ .value_name("QID")
+ .multiple(true)
+ .number_of_values(1),
+ )
+ .arg(
+ Arg::with_name("search")
+ .short("x")
+ .long("search-string")
+ .help("Search for string")
+ .value_name("STRING"),
+ )
+ .arg(
+ Arg::with_name("limit")
+ .short("l")
+ .long("limit")
+ .help("Print MAX entries")
+ .value_name("MAX")
+ .default_value("0"),
+ )
+ .arg(
+ Arg::with_name("exclude_greylist")
+ .short("g")
+ .long("exclude-greylist")
+ .help("Exclude greylist entries"),
+ )
+ .arg(
+ Arg::with_name("exclude_ndr")
+ .short("n")
+ .long("exclude-ndr")
+ .help("Exclude NDR entries"),
+ )
+ .get_matches();
+
+ let mut parser = Parser::new();
+ parser.handle_args(matches)?;
+
+ println!("# LogReader: {}", std::process::id());
+ println!("# Query options");
+ if !parser.options.from.is_empty() {
+ println!("# Sender: {}", parser.options.from);
+ }
+ if !parser.options.to.is_empty() {
+ println!("# Recipient: {}", parser.options.to);
+ }
+ if !parser.options.host.is_empty() {
+ println!("# Server: {}", parser.options.host);
+ }
+ if !parser.options.msgid.is_empty() {
+ println!("# MsgID: {}", parser.options.msgid);
+ }
+ for m in parser.options.match_list.iter() {
+ match m {
+ Match::Qid(b) => println!("# QID: {}", std::str::from_utf8(b)?),
+ Match::RelLineNr(t, l) => println!("# QID: T{:8X}L{:08X}", *t as u32, *l as u32),
+ }
+ }
+
+ if !parser.options.string_match.is_empty() {
+ println!("# Match: {}", parser.options.string_match);
+ }
+
+ println!(
+ "# Start: {} ({})",
+ time::strftime("%F %T", &parser.start_tm)?,
+ parser.options.start
+ );
+ println!(
+ "# End: {} ({})",
+ time::strftime("%F %T", &parser.end_tm)?,
+ parser.options.end
+ );
+
+ println!("# End Query Options\n");
+ parser.parse_files()?;
+
+ Ok(())
+}
+
+fn handle_pmg_smtp_filter_message(msg: &[u8], parser: &mut Parser, complete_line: &[u8]) {
+ let (qid, data) = match parse_qid(msg, 25) {
+ Some((q, m)) => (q, m),
+ None => return,
+ };
+ let data = &data[2..];
+
+ let fe = {
+ if let Some(fe) = parser.fentries.get(qid) {
+ fe.clone()
+ } else {
+ let fe = Rc::new(RefCell::new(FEntry::default()));
+ fe.borrow_mut().logid = qid.to_vec();
+ parser.fentries.insert(qid.to_vec(), fe.clone());
+ fe
+ }
+ };
+
+ if parser.string_match {
+ fe.borrow_mut().string_match = parser.string_match;
+ }
+
+ fe.borrow_mut()
+ .log
+ .push((complete_line.to_vec(), parser.lines));
+
+ if data.starts_with(b"accept mail to <") {
+ let data = &data[16..];
+ let to_count = data.iter().take_while(|b| (**b as char) != '>').count();
+ let (to, data) = data.split_at(to_count);
+ if !data.starts_with(b"> (") {
+ return;
+ }
+ let data = &data[3..];
+ let qid_count = data.iter().take_while(|b| (**b as char) != ')').count();
+ let qid = &data[..qid_count];
+
+ fe.borrow_mut()
+ .set_accept(to, qid, parser.current_record_state.timestamp);
+ return;
+ }
+
+ if data.starts_with(b"moved mail for <") {
+ let data = &data[16..];
+ let to_count = data.iter().take_while(|b| (**b as char) != '>').count();
+ let (to, data) = data.split_at(to_count);
+
+ let qid_index = match find(data, b"quarantine - ") {
+ Some(i) => i,
+ None => return,
+ };
+ let data = &data[qid_index + 13..];
+ let (qid, _) = match parse_qid(data, 25) {
+ Some(t) => t,
+ None => return,
+ };
+
+ fe.borrow_mut()
+ .set_quarantine(to, qid, parser.current_record_state.timestamp);
+ return;
+ }
+
+ if data.starts_with(b"block mail to <") {
+ let data = &data[15..];
+ let to_count = data.iter().take_while(|b| (**b as char) != '>').count();
+ let to = &data[..to_count];
+
+ fe.borrow_mut()
+ .set_block(to, parser.current_record_state.timestamp);
+ return;
+ }
+
+ if data.starts_with(b"processing time: ") {
+ let data = &data[17..];
+ let time_count = data.iter().take_while(|b| !b.is_ascii_whitespace()).count();
+ let time = &data[..time_count];
+
+ fe.borrow_mut().set_processing_time(time);
+ return;
+ }
+}
+
+fn handle_postscreen_message(msg: &[u8], parser: &mut Parser, complete_line: &[u8]) {
+ if !msg.starts_with(b"NOQUEUE: reject: RCPT from ") {
+ return;
+ }
+ let data = &msg[27..];
+ let client_index = match find(data, b"; client [") {
+ Some(i) => i,
+ None => return,
+ };
+ let data = &data[client_index + 10..];
+
+ let client_count = data.iter().take_while(|b| (**b as char) != ']').count();
+ let (client, data) = data.split_at(client_count);
+
+ let from_index = match find(data, b"; from=<") {
+ Some(i) => i,
+ None => return,
+ };
+ let data = &data[from_index + 8..];
+
+ let from_count = data.iter().take_while(|b| (**b as char) != '>').count();
+ let (from, data) = data.split_at(from_count);
+
+ if !data.starts_with(b">, to=<") {
+ return;
+ }
+ let data = &data[7..];
+ let to_count = data.iter().take_while(|b| (**b as char) != '>').count();
+ let to = &data[..to_count];
+
+ let se = {
+ if let Some(se) = parser.sentries.get(&parser.current_record_state.pid) {
+ se.clone()
+ } else {
+ let se = Rc::new(RefCell::new(SEntry::default()));
+ se.borrow_mut().rel_line_nr = parser.rel_line_nr;
+ se.borrow_mut().timestamp = parser.current_record_state.timestamp;
+ parser
+ .sentries
+ .insert(parser.current_record_state.pid, se.clone());
+ se
+ }
+ };
+
+ if parser.string_match {
+ se.borrow_mut().string_match = parser.string_match;
+ }
+
+ se.borrow_mut()
+ .log
+ .push((complete_line.to_vec(), parser.lines));
+ se.borrow_mut().add_noqueue_entry(
+ from,
+ to,
+ DStatus::Noqueue,
+ parser.current_record_state.timestamp,
+ );
+ se.borrow_mut().set_connect(client);
+ se.borrow_mut().disconnect = true;
+ se.borrow_mut().print(parser);
+ parser.free_sentry(parser.current_record_state.pid);
+}
+
+fn handle_qmgr_message(msg: &[u8], parser: &mut Parser, complete_line: &[u8]) {
+ let (qid, data) = match parse_qid(msg, 15) {
+ Some(t) => t,
+ None => return,
+ };
+ let data = &data[2..];
+
+ let qe = {
+ if let Some(qe) = parser.qentries.get(qid) {
+ qe.clone()
+ } else {
+ let qe = Rc::new(RefCell::new(QEntry::default()));
+ qe.borrow_mut().qid = qid.to_vec();
+ parser.qentries.insert(qid.to_vec(), qe.clone());
+ qe
+ }
+ };
+ if parser.string_match {
+ qe.borrow_mut().string_match = parser.string_match;
+ }
+ qe.borrow_mut().cleanup = true;
+ qe.borrow_mut()
+ .log
+ .push((complete_line.to_vec(), parser.lines));
+
+ if data.starts_with(b"from=<") {
+ let data = &data[6..];
+
+ let from_count = data.iter().take_while(|b| (**b as char) != '>').count();
+ let (from, data) = data.split_at(from_count);
+
+ if !data.starts_with(b">, size=") {
+ return;
+ }
+ let data = &data[8..];
+
+ let size_count = data
+ .iter()
+ .take_while(|b| (**b as char).is_ascii_digit())
+ .count();
+ let (size, _) = data.split_at(size_count);
+ qe.borrow_mut().from = from.to_vec();
+ qe.borrow_mut().size = unsafe { std::str::from_utf8_unchecked(size) }
+ .parse()
+ .unwrap();
+ } else if data == b"removed" {
+ qe.borrow_mut().removed = true;
+ qe.borrow_mut().finalize(parser);
+ }
+}
+
+fn handle_lmtp_message(msg: &[u8], parser: &mut Parser, complete_line: &[u8]) {
+ let (qid, data) = match parse_qid(msg, 15) {
+ Some((q, t)) => (q, t),
+ None => return,
+ };
+
+ let qe = {
+ if let Some(qe) = parser.qentries.get(qid) {
+ qe.clone()
+ } else {
+ let qe = Rc::new(RefCell::new(QEntry::default()));
+ qe.borrow_mut().qid = qid.to_vec();
+ parser.qentries.insert(qid.to_vec(), qe.clone());
+ qe
+ }
+ };
+ if parser.string_match {
+ qe.borrow_mut().string_match = parser.string_match;
+ }
+ qe.borrow_mut().cleanup = true;
+ qe.borrow_mut()
+ .log
+ .push((complete_line.to_vec(), parser.lines));
+
+ let data = &data[2..];
+ if !data.starts_with(b"to=<") {
+ return;
+ }
+ let data = &data[4..];
+ let to_count = data.iter().take_while(|b| (**b as char) != '>').count();
+ let (to, data) = data.split_at(to_count);
+
+ let relay_index = match find(data, b"relay=") {
+ Some(i) => i,
+ None => return,
+ };
+ let data = &data[relay_index + 6..];
+ let relay_count = data.iter().take_while(|b| (**b as char) != ',').count();
+ let (relay, data) = data.split_at(relay_count);
+
+ let dsn_index = match find(data, b"dsn=") {
+ Some(i) => i,
+ None => return,
+ };
+ let data = &data[dsn_index + 4..];
+ let dsn = match data.iter().next() {
+ Some(b) => {
+ if (*b as char).is_ascii_digit() {
+ (*b as char).to_digit(10).unwrap()
+ } else {
+ return;
+ }
+ }
+ None => return,
+ };
+
+ qe.borrow_mut().add_to_entry(
+ to,
+ relay,
+ DStatus::Dsn(dsn),
+ parser.current_record_state.timestamp,
+ );
+
+ if parser.current_record_state.sys == b"postfix/lmtp" {
+ let sent_index = match find(data, b"status=sent (250 2.") {
+ Some(i) => i,
+ None => return,
+ };
+ let mut data = &data[sent_index + 19..];
+ if data.starts_with(b"5.0 OK") {
+ data = &data[8..];
+ } else if data.starts_with(b"7.0 BLOCKED") {
+ data = &data[13..];
+ } else {
+ return;
+ }
+
+ let (qid, _) = match parse_qid(data, 25) {
+ Some(t) => t,
+ None => return,
+ };
+
+ qe.borrow_mut().filtered = true;
+ if let Some(fe) = parser.fentries.get(qid) {
+ qe.borrow_mut().filter = Some(fe.clone());
+ }
+ }
+}
+
+fn handle_smtpd_message(msg: &[u8], parser: &mut Parser, complete_line: &[u8]) {
+ let se = {
+ if let Some(se) = parser.sentries.get(&parser.current_record_state.pid) {
+ se.clone()
+ } else {
+ let se = Rc::new(RefCell::new(SEntry::default()));
+ se.borrow_mut().rel_line_nr = parser.rel_line_nr;
+ se.borrow_mut().timestamp = parser.current_record_state.timestamp;
+ parser
+ .sentries
+ .insert(parser.current_record_state.pid, se.clone());
+ se
+ }
+ };
+ if parser.string_match {
+ se.borrow_mut().string_match = parser.string_match;
+ }
+ se.borrow_mut()
+ .log
+ .push((complete_line.to_vec(), parser.lines));
+
+ if msg.starts_with(b"connect from ") {
+ let addr = &msg[13..];
+ se.borrow_mut().set_connect(addr);
+ return;
+ }
+
+ if msg.starts_with(b"disconnect from") {
+ parser.sentries.remove(&parser.current_record_state.pid);
+ se.borrow_mut().disconnect = true;
+
+ if se.borrow_mut().remove_unneeded_refs(parser) == 0 {
+ se.borrow_mut().print(parser);
+ parser.free_sentry(se.borrow().pid);
+ } else {
+ se.borrow_mut().finalize_refs(parser);
+ }
+ return;
+ }
+
+ if msg.starts_with(b"NOQUEUE:") {
+ let data = &msg[8..];
+ let colon_index = match find(data, b":") {
+ Some(i) => i,
+ None => return,
+ };
+ let data = &data[colon_index + 1..];
+ let colon_index = match find(data, b":") {
+ Some(i) => i,
+ None => return,
+ };
+ let data = &data[colon_index + 1..];
+ let semicolon_index = match find(data, b";") {
+ Some(i) => i,
+ None => return,
+ };
+ let (grey, data) = data.split_at(semicolon_index);
+ let dstatus = if find(
+ grey,
+ b"Recipient address rejected: Service is unavailable (try later)",
+ )
+ .is_some()
+ {
+ DStatus::Greylist
+ } else {
+ DStatus::Noqueue
+ };
+
+ if !data.starts_with(b"; from=<") {
+ return;
+ }
+ let data = &data[8..];
+ let from_count = data.iter().take_while(|b| (**b as char) != '>').count();
+ let (from, data) = data.split_at(from_count);
+
+ if !data.starts_with(b"> to=<") {
+ return;
+ }
+ let data = &data[6..];
+ let to_count = data.iter().take_while(|b| (**b as char) != '>').count();
+ let to = &data[..to_count];
+
+ se.borrow_mut()
+ .add_noqueue_entry(from, to, dstatus, parser.current_record_state.timestamp);
+ return;
+ }
+
+ let (qid, data) = match parse_qid(msg, 15) {
+ Some(t) => t,
+ None => return,
+ };
+ let data = &data[2..];
+
+ let qe = {
+ if let Some(qe) = parser.qentries.get(qid) {
+ qe.clone()
+ } else {
+ let qe = Rc::new(RefCell::new(QEntry::default()));
+ qe.borrow_mut().qid = qid.to_vec();
+ parser.qentries.insert(qid.to_vec(), qe.clone());
+ qe
+ }
+ };
+ if parser.string_match {
+ qe.borrow_mut().string_match = parser.string_match;
+ }
+
+ SEntry::add_ref(&se, &qe);
+
+ if !data.starts_with(b"client=") {
+ return;
+ }
+ let data = &data[7..];
+ let client_count = data
+ .iter()
+ .take_while(|b| !(**b as char).is_ascii_whitespace())
+ .count();
+ let client = &data[..client_count];
+
+ qe.borrow_mut().set_client(client);
+}
+
+fn handle_cleanup_message(msg: &[u8], parser: &mut Parser, complete_line: &[u8]) {
+ let (qid, data) = match parse_qid(msg, 15) {
+ Some(t) => t,
+ None => return,
+ };
+ let data = &data[2..];
+
+ let qe = {
+ if let Some(qe) = parser.qentries.get(qid) {
+ qe.clone()
+ } else {
+ let qe = Rc::new(RefCell::new(QEntry::default()));
+ qe.borrow_mut().qid = qid.to_vec();
+ parser.qentries.insert(qid.to_vec(), qe.clone());
+ qe
+ }
+ };
+ if parser.string_match {
+ qe.borrow_mut().string_match = parser.string_match;
+ }
+ qe.borrow_mut()
+ .log
+ .push((complete_line.to_vec(), parser.lines));
+
+ if !data.starts_with(b"message-id=") {
+ return;
+ }
+ let data = &data[11..];
+ let msgid_count = data
+ .iter()
+ .take_while(|b| !(**b as char).is_ascii_whitespace())
+ .count();
+ let msgid = &data[..msgid_count];
+
+ if !msgid.is_empty() {
+ if qe.borrow().msgid.is_empty() {
+ qe.borrow_mut().msgid = msgid.to_vec();
+ }
+ qe.borrow_mut().cleanup = true;
+ }
+}
+
+#[derive(Default, Debug, Clone)]
+struct NoqueueEntry {
+ from: Vec<u8>,
+ to: Vec<u8>,
+ dstatus: DStatus,
+ timestamp: u64,
+}
+
+#[derive(Debug, Clone)]
+struct ToEntry {
+ to: Vec<u8>,
+ relay: Vec<u8>,
+ dstatus: DStatus,
+ timestamp: u64,
+}
+
+impl Default for ToEntry {
+ fn default() -> Self {
+ ToEntry {
+ to: Default::default(),
+ relay: b"none".to_vec(),
+ dstatus: Default::default(),
+ timestamp: Default::default(),
+ }
+ }
+}
+
+#[derive(Debug, PartialEq, Clone)]
+enum DStatus {
+ Invalid,
+ Accept,
+ Quarantine,
+ Block,
+ Greylist,
+ Noqueue,
+ Dsn(u32),
+}
+
+impl Default for DStatus {
+ fn default() -> Self {
+ DStatus::Invalid
+ }
+}
+
+impl std::fmt::Display for DStatus {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ let c = match self {
+ DStatus::Invalid => '\0', // other default
+ DStatus::Accept => 'A',
+ DStatus::Quarantine => 'Q',
+ DStatus::Block => 'B',
+ DStatus::Greylist => 'G',
+ DStatus::Noqueue => 'N',
+ DStatus::Dsn(v) => std::char::from_digit(*v, 10).unwrap(),
+ };
+ write!(f, "{}", c)
+ }
+}
+
+impl DStatus {
+ fn is_dsn(&self, value: Option<u32>) -> bool {
+ match self {
+ DStatus::Dsn(v) => {
+ if let Some(val) = value {
+ *v == val
+ } else {
+ true
+ }
+ }
+ _ => false,
+ }
+ }
+}
+
+#[derive(Debug, Clone, Default)]
+struct SEntry {
+ log: Vec<(Vec<u8>, u64)>,
+ connect: Vec<u8>,
+ cursor: Vec<u8>,
+ pid: u64,
+ refs: Vec<Rc<RefCell<QEntry>>>,
+ nq_entries: Vec<NoqueueEntry>,
+ disconnect: bool,
+ string_match: bool,
+ timestamp: u64,
+ rel_line_nr: u64,
+}
+
+impl SEntry {
+ fn add_noqueue_entry(&mut self, from: &[u8], to: &[u8], dstatus: DStatus, timestamp: u64) {
+ let ne = NoqueueEntry {
+ to: to.to_vec(),
+ from: from.to_vec(),
+ dstatus,
+ timestamp,
+ };
+ self.nq_entries.push(ne);
+ }
+
+ fn set_connect(&mut self, client: &[u8]) {
+ if self.connect.is_empty() {
+ self.connect = client.to_vec();
+ }
+ }
+
+ fn print(&mut self, parser: &mut Parser) {
+ if !parser.options.msgid.is_empty() {
+ return;
+ }
+
+ if !parser.options.host.is_empty() {
+ if self.connect.is_empty() {
+ return;
+ }
+ if find_lowercase(&self.connect, parser.options.host.as_bytes()).is_none() {
+ return;
+ }
+ }
+
+ if !parser.options.match_list.is_empty() {
+ let mut found = false;
+ for m in parser.options.match_list.iter() {
+ match m {
+ Match::Qid(_) => return,
+ Match::RelLineNr(t, l) => {
+ if (*t as u64) == self.timestamp && *l == self.rel_line_nr {
+ found = true;
+ break;
+ }
+ }
+ }
+ }
+ if !found {
+ return;
+ }
+ }
+
+ if !parser.options.from.is_empty()
+ || !parser.options.to.is_empty()
+ || parser.options.exclude_greylist
+ || parser.options.exclude_ndr
+ {
+ let mut found = false;
+ for nq in self.nq_entries.iter_mut().rev() {
+ if !parser.options.from.is_empty()
+ && find_lowercase(&nq.from, parser.options.from.as_bytes()).is_none()
+ {
+ nq.dstatus = DStatus::Invalid;
+ }
+
+ if parser.options.exclude_greylist && nq.dstatus == DStatus::Greylist {
+ nq.dstatus = DStatus::Invalid;
+ }
+ if parser.options.exclude_ndr && nq.from.is_empty() {
+ nq.dstatus = DStatus::Invalid;
+ }
+
+ if !parser.options.to.is_empty()
+ && !nq.to.is_empty()
+ && find_lowercase(&nq.to, parser.options.to.as_bytes()).is_none()
+ {
+ nq.dstatus = DStatus::Invalid;
+ }
+
+ if nq.dstatus != DStatus::Invalid {
+ found = true;
+ }
+ }
+
+ if !found {
+ return;
+ }
+ }
+
+ if !parser.options.string_match.is_empty() && !self.string_match {
+ return;
+ }
+
+ if parser.options.verbose > 0 {
+ parser.write_all_ok(format!(
+ "SMTPD: T{:8X}L{:08X}\n",
+ self.timestamp as u32, self.rel_line_nr as u32
+ ));
+ parser.write_all_ok(format!("CTIME: {:8X}\n", parser.ctime).as_bytes());
+
+ if !self.connect.is_empty() {
+ parser.write_all_ok(b"CLIENT: ");
+ parser.write_all_ok(&self.connect);
+ parser.write_all_ok(b"\n");
+ }
+ }
+
+ for nq in self.nq_entries.iter().rev() {
+ if nq.dstatus != DStatus::Invalid {
+ parser.write_all_ok(format!(
+ "TO:{:X}:T{:08X}L{:08X}:{}: from <", //{}> to <{}>",
+ nq.timestamp as i32, self.timestamp as i32, self.rel_line_nr, nq.dstatus,
+ ));
+ parser.write_all_ok(&nq.from);
+ parser.write_all_ok(b"> to <");
+ parser.write_all_ok(&nq.to);
+ parser.write_all_ok(b">\n");
+ parser.count += 1;
+ }
+ }
+
+ if parser.options.verbose > 1 {
+ parser.write_all_ok(b"LOGS:\n");
+ for (log, line) in self.log.iter() {
+ parser.write_all_ok(format!("L{:08X} ", *line as u32));
+ parser.write_all_ok(log);
+ parser.write_all_ok(b"\n");
+ }
+ }
+ parser.write_all_ok(b"\n");
+ }
+
+ fn delete_ref(&mut self, qentry: &QEntry) -> u32 {
+ let mut count: u32 = 0;
+ self.refs.retain(|q| {
+ if std::ptr::eq(&*q.borrow(), qentry) {
+ return false;
+ } else if qentry.cleanup {
+ count += 1;
+ }
+ true
+ });
+ count
+ }
+
+ fn remove_unneeded_refs(&mut self, parser: &mut Parser) -> u32 {
+ let mut count: u32 = 0;
+ self.refs.retain(|q| {
+ let is_cleanup = q.borrow().cleanup;
+ if !is_cleanup {
+ q.borrow_mut().smtpd = None;
+ parser.free_qentry(&q.borrow().qid);
+ false
+ } else {
+ count += 1;
+ true
+ }
+ });
+ count
+ }
+
+ fn finalize_refs(&mut self, parser: &mut Parser) {
+ //let mut count: u32 = 0;
+ let mut qentries = Vec::new();
+ for q in self.refs.iter() {
+ //count += 1;
+ if !q.borrow().removed {
+ continue;
+ }
+
+ let fe = q.borrow().filter.clone();
+ if let Some(f) = fe {
+ if !f.borrow().finished {
+ continue;
+ }
+ }
+
+ //count -= 1;
+ qentries.push(q.clone());
+ }
+
+ for q in qentries.iter() {
+ q.borrow_mut().print(parser, Some(self));
+ q.borrow_mut().smtpd = None;
+ parser.free_qentry(&q.borrow().qid);
+
+ if let Some(f) = q.borrow().filter.clone() {
+ parser.free_fentry(&f.borrow().logid);
+ }
+ }
+ }
+
+ fn add_ref(sentry: &Rc<RefCell<SEntry>>, qentry: &Rc<RefCell<QEntry>>) {
+ let smtpd = qentry.borrow().smtpd.clone();
+ if smtpd.is_some() {
+ if let Some(s) = smtpd {
+ if !Rc::ptr_eq(sentry, &s) {
+ eprintln!("Error: qentry ref already set");
+ }
+ }
+ return;
+ }
+
+ for q in sentry.borrow().refs.iter() {
+ if Rc::ptr_eq(q, qentry) {
+ return;
+ }
+ }
+
+ sentry.borrow_mut().refs.push(qentry.clone());
+ qentry.borrow_mut().smtpd = Some(sentry.clone());
+ }
+}
+
+#[derive(Default, Debug, Clone)]
+struct QEntry {
+ log: Vec<(Vec<u8>, u64)>,
+ smtpd: Option<Rc<RefCell<SEntry>>>,
+ filter: Option<Rc<RefCell<FEntry>>>,
+ qid: Vec<u8>,
+ from: Vec<u8>,
+ client: Vec<u8>,
+ msgid: Vec<u8>,
+ size: u64,
+ to_entries: Vec<ToEntry>,
+ cleanup: bool,
+ removed: bool,
+ filtered: bool,
+ string_match: bool,
+}
+
+impl QEntry {
+ fn add_to_entry(&mut self, to: &[u8], relay: &[u8], dstatus: DStatus, timestamp: u64) {
+ let te = ToEntry {
+ to: to.to_vec(),
+ relay: relay.to_vec(),
+ dstatus,
+ timestamp,
+ };
+ self.to_entries.push(te);
+ }
+
+ fn finalize(&mut self, parser: &mut Parser) {
+ if self.removed {
+ if let Some(se) = self.smtpd.clone() {
+ if !se.borrow().disconnect {
+ return;
+ }
+ }
+
+ if let Some(fe) = self.filter.clone() {
+ if !fe.borrow().finished {
+ return;
+ }
+
+ match self.smtpd.clone() {
+ Some(s) => self.print(parser, Some(&*s.borrow())),
+ None => self.print(parser, None),
+ };
+ parser.free_qentry(&self.qid);
+
+ parser.free_fentry(&fe.borrow().logid);
+ } else {
+ match self.smtpd.clone() {
+ Some(s) => self.print(parser, Some(&*s.borrow())),
+ None => self.print(parser, None),
+ };
+ parser.free_qentry(&self.qid);
+ }
+ }
+ }
+
+ fn msgid_matches(&self, parser: &Parser) -> bool {
+ if !parser.options.msgid.is_empty() {
+ if self.msgid.is_empty() {
+ return false;
+ }
+ let qentry_msgid_lowercase = self.msgid.to_ascii_lowercase();
+ let msgid_lowercase = parser.options.msgid.as_bytes().to_ascii_lowercase();
+ if qentry_msgid_lowercase != msgid_lowercase {
+ return false;
+ }
+ }
+ true
+ }
+
+ fn match_list_matches(&self, parser: &Parser, se: Option<&SEntry>) -> bool {
+ let fe = self.filter.clone();
+ if !parser.options.match_list.is_empty() {
+ let mut found = false;
+ for m in parser.options.match_list.iter() {
+ match m {
+ Match::Qid(q) => {
+ if let Some(f) = fe.clone() {
+ if &f.borrow().logid == q {
+ found = true;
+ break;
+ }
+ }
+ if &self.qid == q {
+ found = true;
+ break;
+ }
+ }
+ Match::RelLineNr(t, l) => {
+ if let Some(s) = se {
+ if s.timestamp == (*t as u64) && s.rel_line_nr == *l {
+ found = true;
+ break;
+ }
+ }
+ }
+ }
+ }
+ if !found {
+ return false;
+ }
+ }
+ true
+ }
+
+ fn host_matches(&self, parser: &Parser, se: Option<&SEntry>) -> bool {
+ if !parser.options.host.is_empty() {
+ let mut found = false;
+ if let Some(s) = se {
+ if !s.connect.is_empty()
+ && find_lowercase(&s.connect, parser.options.host.as_bytes()).is_some()
+ {
+ found = true;
+ }
+ }
+ if !self.client.is_empty()
+ && find_lowercase(&self.client, parser.options.host.as_bytes()).is_some()
+ {
+ found = true;
+ }
+
+ if !found {
+ return false;
+ }
+ }
+ true
+ }
+
+ fn from_to_matches(&mut self, parser: &Parser) -> bool {
+ if !parser.options.from.is_empty() {
+ if self.from.is_empty() {
+ return false;
+ }
+ if find_lowercase(&self.from, parser.options.from.as_bytes()).is_none() {
+ return false;
+ }
+ } else if parser.options.exclude_ndr && self.from.is_empty() {
+ return false;
+ }
+
+ if !parser.options.to.is_empty() {
+ let mut found = false;
+ self.to_entries.retain(|to| {
+ if find_lowercase(&to.to, parser.options.to.as_bytes()).is_none() {
+ false
+ } else {
+ found = true;
+ true
+ }
+ });
+ if !found {
+ return false;
+ }
+ }
+ true
+ }
+
+ fn string_matches(&self, parser: &Parser, se: Option<&SEntry>) -> bool {
+ let fe = self.filter.clone();
+ if !parser.options.string_match.is_empty() {
+ let mut string_match = self.string_match;
+
+ if let Some(s) = se {
+ if s.string_match {
+ string_match = true;
+ }
+ }
+ if let Some(f) = fe.clone() {
+ if f.borrow().string_match {
+ string_match = true;
+ }
+ }
+ if !string_match {
+ return false;
+ }
+ }
+ true
+ }
+
+ fn print(&mut self, parser: &mut Parser, se: Option<&SEntry>) {
+ let fe = self.filter.clone();
+
+ if !self.msgid_matches(parser) {
+ return;
+ }
+
+ if !self.match_list_matches(parser, se) {
+ return;
+ }
+
+ if !self.host_matches(parser, se) {
+ return;
+ }
+
+ if !self.from_to_matches(parser) {
+ return;
+ }
+
+ if !self.string_matches(parser, se) {
+ return;
+ }
+
+ if parser.options.verbose > 0 {
+ parser.write_all_ok(b"QENTRY: ");
+ parser.write_all_ok(&self.qid);
+ parser.write_all_ok(b"\n");
+ parser.write_all_ok(format!("CTIME: {:8X}\n", parser.ctime));
+ parser.write_all_ok(format!("SIZE: {}\n", self.size));
+
+ if !self.client.is_empty() {
+ parser.write_all_ok(b"CLIENT: ");
+ parser.write_all_ok(&self.client);
+ parser.write_all_ok(b"\n");
+ } else if let Some(s) = se {
+ if !s.connect.is_empty() {
+ parser.write_all_ok(b"CLIENT: ");
+ parser.write_all_ok(&s.connect);
+ parser.write_all_ok(b"\n");
+ }
+ }
+
+ if !self.msgid.is_empty() {
+ parser.write_all_ok(b"MSGID: ");
+ parser.write_all_ok(&self.msgid);
+ parser.write_all_ok(b"\n");
+ }
+ }
+
+ for to in self.to_entries.iter().rev() {
+ if !to.to.is_empty() {
+ let mut final_to = to.clone();
+ if to.dstatus.is_dsn(Some(2)) {
+ if let Some(f) = fe.clone() {
+ for to2 in f.borrow().to_entries.iter().rev() {
+ if to.to == to2.to {
+ final_to = to2.clone();
+ break;
+ }
+ }
+ }
+ }
+
+ parser.write_all_ok(format!("TO:{:X}:", to.timestamp as i32,));
+ parser.write_all_ok(&self.qid);
+ parser.write_all_ok(format!(":{}: from <", final_to.dstatus));
+ parser.write_all_ok(&self.from);
+ parser.write_all_ok(b"> to <");
+ parser.write_all_ok(&final_to.to);
+ parser.write_all_ok(b"> (");
+ parser.write_all_ok(&final_to.relay);
+ parser.write_all_ok(b")\n");
+ parser.count += 1;
+ }
+ }
+
+ if parser.options.verbose > 1 {
+ let print_log = |parser: &mut Parser, logs: &Vec<(Vec<u8>, u64)>| {
+ for (log, line) in logs.iter() {
+ parser.write_all_ok(format!("L{:08X} ", *line as u32));
+ parser.write_all_ok(log);
+ parser.write_all_ok(b"\n");
+ }
+ };
+ if let Some(s) = se {
+ if !s.log.is_empty() {
+ parser.write_all_ok(b"SMTP:\n");
+ print_log(parser, &s.log);
+ }
+ }
+
+ if let Some(f) = fe {
+ if !f.borrow().log.is_empty() {
+ parser.write_all_ok(format!("FILTER: {}\n", unsafe {
+ std::str::from_utf8_unchecked(&f.borrow().logid)
+ }));
+ print_log(parser, &f.borrow().log);
+ }
+ }
+
+ if !self.log.is_empty() {
+ parser.write_all_ok(b"QMGR:\n");
+ print_log(parser, &self.log);
+ }
+ }
+ parser.write_all_ok(b"\n")
+ }
+
+ fn set_client(&mut self, client: &[u8]) {
+ if self.client.is_empty() {
+ self.client = client.to_vec();
+ }
+ }
+}
+
+impl Drop for QEntry {
+ fn drop(&mut self) {
+ if let Some(se) = self.smtpd.take() {
+ let count = se.borrow_mut().delete_ref(self);
+ if count == 0 && se.borrow().disconnect {
+ // drop(se);
+ }
+ }
+ }
+}
+
+#[derive(Default, Debug, Clone)]
+struct FEntry {
+ log: Vec<(Vec<u8>, u64)>,
+ logid: Vec<u8>,
+ to_entries: Vec<ToEntry>,
+ processing_time: Vec<u8>,
+ string_match: bool,
+ finished: bool,
+}
+
+impl FEntry {
+ fn set_accept(&mut self, to: &[u8], qid: &[u8], timestamp: u64) {
+ let te = ToEntry {
+ to: to.to_vec(),
+ relay: qid.to_vec(),
+ dstatus: DStatus::Accept,
+ timestamp,
+ };
+ self.to_entries.push(te);
+ }
+
+ fn set_quarantine(&mut self, to: &[u8], qid: &[u8], timestamp: u64) {
+ let te = ToEntry {
+ to: to.to_vec(),
+ relay: qid.to_vec(),
+ dstatus: DStatus::Quarantine,
+ timestamp,
+ };
+ self.to_entries.push(te);
+ }
+
+ fn set_block(&mut self, to: &[u8], timestamp: u64) {
+ let te = ToEntry {
+ to: to.to_vec(),
+ relay: b"none".to_vec(),
+ dstatus: DStatus::Block,
+ timestamp,
+ };
+ self.to_entries.push(te);
+ }
+
+ fn set_processing_time(&mut self, time: &[u8]) {
+ self.processing_time = time.to_vec();
+ self.finished = true;
+ }
+}
+
+#[derive(Debug)]
+struct Parser {
+ sentries: HashMap<u64, Rc<RefCell<SEntry>>>,
+ fentries: HashMap<Vec<u8>, Rc<RefCell<FEntry>>>,
+ qentries: HashMap<Vec<u8>, Rc<RefCell<QEntry>>>,
+
+ current_record_state: RecordState,
+ rel_line_nr: u64,
+
+ current_year: [i64; 32],
+ current_month: i64,
+ current_file_index: usize,
+
+ count: u64,
+
+ buffered_stdout: BufWriter<std::io::Stdout>,
+
+ options: Options,
+
+ start_tm: time::Tm,
+ end_tm: time::Tm,
+
+ ctime: libc::time_t,
+ string_match: bool,
+
+ lines: u64,
+}
+
+impl Parser {
+ fn new() -> Self {
+ let mut years: [i64; 32] = [0; 32];
+ let mut tv: libc::timeval = libc::timeval {
+ tv_sec: 0,
+ tv_usec: 0,
+ };
+ let mut ltime: *mut libc::tm;
+
+ for (i, year) in years.iter_mut().enumerate() {
+ unsafe {
+ libc::gettimeofday(&mut tv, std::ptr::null_mut());
+ }
+ tv.tv_sec -= (3600 * 24 * i) as i64;
+ ltime = unsafe { libc::localtime(&tv.tv_sec) };
+ *year = (unsafe { (*ltime).tm_year + 1900 }) as i64;
+ }
+
+ Self {
+ sentries: HashMap::new(),
+ fentries: HashMap::new(),
+ qentries: HashMap::new(),
+ current_record_state: Default::default(),
+ rel_line_nr: 0,
+ current_year: years,
+ current_month: 0,
+ current_file_index: 0,
+ count: 0,
+ buffered_stdout: BufWriter::new(std::io::stdout()),
+ options: Options::default(),
+ start_tm: time::empty_tm(),
+ end_tm: time::empty_tm(),
+ ctime: 0,
+ string_match: false,
+ lines: 0,
+ }
+ }
+
+ fn free_sentry(&mut self, sentry_pid: u64) {
+ self.sentries.remove(&sentry_pid);
+ }
+
+ fn free_qentry(&mut self, qentry_qid: &[u8]) {
+ self.qentries.remove(qentry_qid);
+ }
+
+ fn free_fentry(&mut self, fentry_logid: &[u8]) {
+ self.fentries.remove(fentry_logid);
+ }
+
+ fn parse_files(&mut self) -> Result<(), Error> {
+ if !self.options.inputfile.is_empty() {
+ if self.options.inputfile == "-" {
+ self.current_file_index = 0;
+ let mut reader = BufReader::new(std::io::stdin());
+ self.handle_input_by_line(&mut reader)?;
+ } else if let Ok(file) = File::open(&self.options.inputfile) {
+ self.current_file_index = 0;
+ let mut reader = BufReader::new(file);
+ self.handle_input_by_line(&mut reader)?;
+ }
+ } else {
+ let filecount = self.count_relevant_files();
+ for i in (0..filecount).rev() {
+ self.current_month = 0;
+ if let Ok(file) = File::open(LOGFILES[i]) {
+ self.current_file_index = i;
+ if i > 1 {
+ let gzdecoder = read::GzDecoder::new(file);
+ let mut reader = BufReader::new(gzdecoder);
+ self.handle_input_by_line(&mut reader)?;
+ } else {
+ let mut reader = BufReader::new(file);
+ self.handle_input_by_line(&mut reader)?;
+ }
+ }
+ }
+ }
+
+ Ok(())
+ }
+
+ fn handle_input_by_line(&mut self, reader: &mut dyn BufRead) -> Result<(), Error> {
+ let mut buffer = Vec::<u8>::with_capacity(4096);
+ let mut prev_time = 0;
+ loop {
+ if self.options.limit > 0 && (self.count >= self.options.limit) {
+ self.write_all_ok("STATUS: aborted by limit (too many hits)\n");
+ self.buffered_stdout.flush()?;
+ std::process::exit(0);
+ }
+
+ buffer.clear();
+ let size = match reader.read_until(b'\n', &mut buffer) {
+ Err(e) => return Err(e.into()),
+ Ok(0) => return Ok(()),
+ Ok(s) => s,
+ };
+ let line = &buffer[0..size - 1];
+ let complete_line = line;
+
+ let (time, line) = match parse_time(
+ line,
+ self.current_year[self.current_file_index],
+ &mut self.current_month,
+ ) {
+ Some(t) => t,
+ None => continue,
+ };
+ if time != prev_time {
+ self.rel_line_nr = 0;
+ } else {
+ self.rel_line_nr += 1;
+ }
+ prev_time = time;
+
+ if time < self.options.start {
+ continue;
+ }
+ if time > self.options.end {
+ break;
+ }
+
+ self.lines += 1;
+
+ let (host, service, pid, line) = match parse_host_service_pid(line) {
+ Some((h, s, p, l)) => (h, s, p, l),
+ None => continue,
+ };
+
+ self.ctime = time;
+
+ self.current_record_state.host = host.to_vec();
+ self.current_record_state.cursor = b"".to_vec();
+ self.current_record_state.sys = service.to_vec();
+ self.current_record_state.pid = pid;
+ self.current_record_state.timestamp = time as u64;
+
+ self.string_match = false;
+ if !self.options.string_match.is_empty()
+ && find(complete_line, self.options.string_match.as_bytes()).is_some()
+ {
+ self.string_match = true;
+ eprintln!("{}", std::str::from_utf8(complete_line)?);
+ }
+
+ if service == b"pmg-smtp-filter" {
+ handle_pmg_smtp_filter_message(line, self, complete_line);
+ } else if service == b"postfix/postscreen" {
+ handle_postscreen_message(line, self, complete_line);
+ } else if service == b"postfix/qmgr" {
+ handle_qmgr_message(line, self, complete_line);
+ } else if service == b"postfix/lmtp"
+ || service == b"postfix/smtp"
+ || service == b"postfix/local"
+ || service == b"postfix/error"
+ {
+ handle_lmtp_message(line, self, complete_line);
+ } else if service == b"postfix/smtpd" {
+ handle_smtpd_message(line, self, complete_line);
+ } else if service == b"postfix/cleanup" {
+ handle_cleanup_message(line, self, complete_line);
+ }
+ }
+ Ok(())
+ }
+
+ /// Returns the number of files to parse. Does not error out if it can't access any file
+ /// (permission denied)
+ fn count_relevant_files(&mut self) -> usize {
+ let mut count = 0;
+ let mut buffer = Vec::new();
+
+ for i in 0..LOGFILES.len() {
+ self.current_month = 0;
+
+ count = i;
+ if let Ok(file) = File::open(LOGFILES[i]) {
+ self.current_file_index = i;
+ buffer.clear();
+ if i > 1 {
+ let gzdecoder = read::GzDecoder::new(file);
+ let mut reader = BufReader::new(gzdecoder);
+ if let Ok(size) = reader.read_until(b'\n', &mut buffer) {
+ if size == 0 {
+ return count;
+ }
+ if let Some((time, _)) = parse_time(
+ &buffer[0..size],
+ self.current_year[i],
+ &mut self.current_month,
+ ) {
+ if time < self.options.start {
+ break;
+ }
+ }
+ } else {
+ return count;
+ }
+ } else {
+ let mut reader = BufReader::new(file);
+ if let Ok(size) = reader.read_until(b'\n', &mut buffer) {
+ if size == 0 {
+ return count;
+ }
+ if let Some((time, _)) = parse_time(
+ &buffer[0..size],
+ self.current_year[i],
+ &mut self.current_month,
+ ) {
+ if time < self.options.start {
+ break;
+ }
+ }
+ } else {
+ return count;
+ }
+ }
+ } else {
+ return count;
+ }
+ }
+
+ count + 1
+ }
+
+ fn handle_args(&mut self, args: clap::ArgMatches) -> Result<(), Error> {
+ if let Some(inputfile) = args.value_of("inputfile") {
+ self.options.inputfile = inputfile.to_string();
+ }
+
+ if let Some(start) = args.value_of("start") {
+ if let Ok(res) = time::strptime(&start, "%F %T") {
+ self.options.start = mkgmtime(&res);
+ self.start_tm = res;
+ } else if let Ok(res) = time::strptime(&start, "%s") {
+ self.options.start = mkgmtime(&res);
+ self.start_tm = res;
+ } else {
+ failure::bail!(failure::err_msg("failed to parse start time"));
+ }
+ } else {
+ let mut ltime = time::now();
+ ltime.tm_sec = 0;
+ ltime.tm_min = 0;
+ ltime.tm_hour = 0;
+ self.options.start = mkgmtime(<ime);
+ self.start_tm = ltime;
+ }
+
+ if let Some(end) = args.value_of("end") {
+ if let Ok(res) = time::strptime(&end, "%F %T") {
+ self.options.end = mkgmtime(&res);
+ self.end_tm = res;
+ } else if let Ok(res) = time::strptime(&end, "%s") {
+ self.options.end = mkgmtime(&res);
+ self.end_tm = res;
+ } else {
+ failure::bail!(failure::err_msg("failed to parse end time"));
+ }
+ } else {
+ let ltime = time::now();
+ self.options.end = mkgmtime(<ime);
+ self.end_tm = ltime;
+ }
+
+ if self.options.end < self.options.start {
+ failure::bail!(failure::err_msg("end time before start time"));
+ }
+
+ self.options.limit = match args.value_of("limit") {
+ Some(l) => l.parse().unwrap(),
+ None => 0,
+ };
+
+ if let Some(qids) = args.values_of("qids") {
+ for q in qids {
+ let ltime: libc::time_t = 0;
+ let rel_line_nr: libc::c_ulong = 0;
+ let input = CString::new(q)?;
+ let bytes = concat!("T%08lXL%08lX", "\0");
+ let format =
+ unsafe { std::ffi::CStr::from_bytes_with_nul_unchecked(bytes.as_bytes()) };
+ if unsafe {
+ libc::sscanf(input.as_ptr(), format.as_ptr(), <ime, &rel_line_nr) == 2
+ } {
+ self.options
+ .match_list
+ .push(Match::RelLineNr(ltime, rel_line_nr));
+ } else {
+ self.options
+ .match_list
+ .push(Match::Qid(q.as_bytes().to_vec()));
+ }
+ }
+ }
+
+ if let Some(from) = args.value_of("from") {
+ self.options.from = from.to_string();
+ }
+ if let Some(to) = args.value_of("to") {
+ self.options.to = to.to_string();
+ }
+ if let Some(host) = args.value_of("host") {
+ self.options.host = host.to_string();
+ }
+ if let Some(msgid) = args.value_of("msgid") {
+ self.options.msgid = msgid.to_string();
+ }
+
+ self.options.exclude_greylist = args.is_present("exclude_greylist");
+ self.options.exclude_ndr = args.is_present("exclude_ndr");
+
+ self.options.verbose = args.occurrences_of("verbose") as _;
+
+ if let Some(string_match) = args.value_of("search") {
+ self.options.string_match = string_match.to_string();
+ }
+
+ Ok(())
+ }
+
+ fn write_all_ok<T: AsRef<[u8]>>(&mut self, data: T) {
+ self.buffered_stdout
+ .write_all(data.as_ref())
+ .expect("failed to write to stdout");
+ }
+}
+
+impl Drop for Parser {
+ fn drop(&mut self) {
+ let qentries = std::mem::replace(&mut self.qentries, HashMap::new());
+ for q in qentries.values() {
+ let smtpd = q.borrow().smtpd.clone();
+ match smtpd {
+ Some(s) => {
+ q.borrow_mut().print(self, Some(&*s.borrow()));
+ }
+ None => {
+ q.borrow_mut().print(self, None);
+ }
+ };
+ }
+ let sentries = std::mem::replace(&mut self.sentries, HashMap::new());
+ for s in sentries.values() {
+ s.borrow_mut().print(self);
+ }
+ }
+}
+
+fn mkgmtime(tm: &time::Tm) -> libc::time_t {
+ let mut res: libc::time_t;
+
+ let mut year = (tm.tm_year + 1900) as i64;
+ let mon = tm.tm_mon;
+
+ res = (year - 1970) * 365 + CAL_MTOD[mon as usize];
+
+ if mon <= 1 {
+ year -= 1;
+ }
+
+ res += (year - 1968) / 4;
+ res -= (year - 1900) / 100;
+ res += (year - 1600) / 400;
+
+ res += (tm.tm_mday - 1) as i64;
+ res = res * 24 + tm.tm_hour as i64;
+ res = res * 60 + tm.tm_min as i64;
+ res = res * 60 + tm.tm_sec as i64;
+
+ res
+}
+
+#[derive(Debug, Default, Clone)]
+struct Options {
+ match_list: Vec<Match>,
+ inputfile: String,
+ string_match: String,
+ host: String,
+ msgid: String,
+ from: String,
+ to: String,
+ start: libc::time_t,
+ end: libc::time_t,
+ limit: u64,
+ verbose: u32,
+ exclude_greylist: bool,
+ exclude_ndr: bool,
+}
+
+#[derive(Debug, Clone)]
+enum Match {
+ Qid(Vec<u8>),
+ RelLineNr(libc::time_t, u64),
+}
+
+#[derive(Debug, Default, Clone)]
+struct RecordState {
+ host: Vec<u8>,
+ cursor: Vec<u8>,
+ sys: Vec<u8>,
+ pid: u64,
+ timestamp: u64,
+}
+
+const LOGFILES: [&str; 32] = [
+ "/var/log/syslog",
+ "/var/log/syslog.1",
+ "/var/log/syslog.2.gz",
+ "/var/log/syslog.3.gz",
+ "/var/log/syslog.4.gz",
+ "/var/log/syslog.5.gz",
+ "/var/log/syslog.6.gz",
+ "/var/log/syslog.7.gz",
+ "/var/log/syslog.8.gz",
+ "/var/log/syslog.9.gz",
+ "/var/log/syslog.10.gz",
+ "/var/log/syslog.11.gz",
+ "/var/log/syslog.12.gz",
+ "/var/log/syslog.13.gz",
+ "/var/log/syslog.14.gz",
+ "/var/log/syslog.15.gz",
+ "/var/log/syslog.16.gz",
+ "/var/log/syslog.17.gz",
+ "/var/log/syslog.18.gz",
+ "/var/log/syslog.19.gz",
+ "/var/log/syslog.20.gz",
+ "/var/log/syslog.21.gz",
+ "/var/log/syslog.22.gz",
+ "/var/log/syslog.23.gz",
+ "/var/log/syslog.24.gz",
+ "/var/log/syslog.25.gz",
+ "/var/log/syslog.26.gz",
+ "/var/log/syslog.27.gz",
+ "/var/log/syslog.28.gz",
+ "/var/log/syslog.29.gz",
+ "/var/log/syslog.30.gz",
+ "/var/log/syslog.31.gz",
+];
+
+/// Parse a number. Returns a tuple of (parsed_number, remaining_text) or None.
+fn parse_number(data: &[u8], max_digits: usize) -> Option<(usize, &[u8])> {
+ let mut bytes = data.iter();
+ let mut number: usize = 0;
+ let mut digits: usize = 0;
+ while digits < max_digits {
+ let c = match bytes.next() {
+ Some(c) => c,
+ None => break,
+ };
+ if (*c as char).is_digit(10) {
+ digits += 1;
+ number *= 10;
+ number += (*c as char).to_digit(10).unwrap() as usize;
+ } else {
+ break;
+ }
+ }
+ if digits == 0 {
+ None
+ } else {
+ Some((number, &data[digits..]))
+ }
+}
+
+/// Parse time. Returns a tuple of (parsed_time, remaining_text) or None.
+fn parse_time<'a>(
+ data: &'a [u8],
+ cur_year: i64,
+ cur_month: &mut i64,
+) -> Option<(libc::time_t, &'a [u8])> {
+ if data.len() < 15 {
+ return None;
+ }
+
+ let mon = match &data[0..3] {
+ b"Jan" => 0,
+ b"Feb" => 1,
+ b"Mar" => 2,
+ b"Apr" => 3,
+ b"May" => 4,
+ b"Jun" => 5,
+ b"Jul" => 6,
+ b"Aug" => 7,
+ b"Sep" => 8,
+ b"Oct" => 9,
+ b"Nov" => 10,
+ b"Dec" => 11,
+ _ => return None,
+ };
+ let data = &data[3..];
+
+ let mut ltime: libc::time_t;
+ let mut year = cur_year;
+
+ if *cur_month == 11 && mon == 0 {
+ year += 1;
+ }
+ if mon > *cur_month {
+ *cur_month = mon;
+ }
+
+ ltime = (year - 1970) * 365 + CAL_MTOD[mon as usize];
+
+ if mon <= 1 {
+ year -= 1;
+ }
+
+ ltime += (year - 1968) / 4;
+ ltime -= (year - 1900) / 100;
+ ltime += (year - 1600) / 400;
+
+ let whitespace_count = data.iter().take_while(|b| b.is_ascii_whitespace()).count();
+ let data = &data[whitespace_count..];
+
+ let (mday, data) = match parse_number(data, 2) {
+ Some(t) => t,
+ None => {
+ eprintln!("no day matched");
+ return None;
+ }
+ };
+ if mday == 0 {
+ eprintln!("mday == 0");
+ return None;
+ }
+
+ ltime += (mday - 1) as i64;
+
+ let data = &data[1..];
+
+ let (hour, data) = match parse_number(data, 2) {
+ Some(t) => t,
+ None => {
+ eprintln!("no hour matched");
+ return None;
+ }
+ };
+
+ ltime *= 24;
+ ltime += hour as i64;
+
+ if let Some(c) = data.iter().next() {
+ if (*c as char) != ':' {
+ eprintln!("char != ':'");
+ return None;
+ }
+ } else {
+ eprintln!("no next char");
+ return None;
+ }
+ let data = &data[1..];
+
+ let (min, data) = match parse_number(data, 2) {
+ Some(t) => t,
+ None => {
+ eprintln!("no min matched");
+ return None;
+ }
+ };
+
+ ltime *= 60;
+ ltime += min as i64;
+
+ if let Some(c) = data.iter().next() {
+ if (*c as char) != ':' {
+ eprintln!("char != ':'");
+ return None;
+ }
+ } else {
+ eprintln!("no next char");
+ return None;
+ }
+ let data = &data[1..];
+
+ let (sec, data) = match parse_number(data, 2) {
+ Some(t) => t,
+ None => {
+ eprintln!("no sec matched");
+ return None;
+ }
+ };
+
+ ltime *= 60;
+ ltime += sec as i64;
+
+ let data = &data[1..];
+
+ Some((ltime, data))
+}
+
+const CAL_MTOD: [i64; 12] = [0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334];
+
+type ByteSlice<'a> = &'a [u8];
+/// Parse Host, Service and PID at the beginning of data. Returns a tuple of (host, service, pid, remaining_text).
+fn parse_host_service_pid(data: &[u8]) -> Option<(ByteSlice, ByteSlice, u64, ByteSlice)> {
+ let host_count = data
+ .iter()
+ .take_while(|b| !(**b as char).is_ascii_whitespace())
+ .count();
+ let host = &data[0..host_count];
+ let data = &data[host_count + 1..]; // whitespace after host
+
+ let service_count = data
+ .iter()
+ .take_while(|b| {
+ (**b as char).is_ascii_alphabetic() || (**b as char) == '/' || (**b as char) == '-'
+ })
+ .count();
+ let service = &data[0..service_count];
+ let data = &data[service_count..];
+ if data.get(0) != Some(&b'[') {
+ return None;
+ }
+ let data = &data[1..];
+
+ let pid_count = data
+ .iter()
+ .take_while(|b| (**b as char).is_ascii_digit())
+ .count();
+ let pid = match unsafe { std::str::from_utf8_unchecked(&data[0..pid_count]) }.parse() {
+ // all ascii digits so valid utf8
+ Ok(p) => p,
+ Err(e) => {
+ eprintln!("failed to parse PID: {}", e);
+ return None;
+ }
+ };
+ let data = &data[pid_count..];
+ if !data.starts_with(b"]: ") {
+ eprintln!("error after PID");
+ return None;
+ }
+ let data = &data[3..];
+
+ Some((host, service, pid, data))
+}
+
+/// A find implementation for [u8]. Returns the index or None.
+fn find<T: PartialOrd>(data: &[T], needle: &[T]) -> Option<usize> {
+ data.windows(needle.len()).position(|d| d == needle)
+}
+
+/// A find implementation for [u8] that converts to lowercase before the comparison. Returns the
+/// index or None.
+fn find_lowercase(data: &[u8], needle: &[u8]) -> Option<usize> {
+ let data = data.to_ascii_lowercase();
+ let needle = needle.to_ascii_lowercase();
+ data.windows(needle.len()).position(|d| d == &needle[..])
+}
diff --git a/src/pmg-log-tracker.c b/src/pmg-log-tracker.c
deleted file mode 100644
index b4ba612..0000000
--- a/src/pmg-log-tracker.c
+++ /dev/null
@@ -1,2580 +0,0 @@
-/*
-
- (C) 2007-2017 Proxmox Server Solutions GmbH, All Rights Reserved
-
- Proxmox Mail Tracker
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU Affero General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU Affero General Public License for more details.
-
- You should have received a copy of the GNU Affero General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>.
-
- Author: Dietmar Maurer <dietmar at proxmox.com>
-
- See http://www.proxmox.com for more information
-
-*/
-
-#define _GNU_SOURCE
-
-#include <stdlib.h>
-#include <stdio.h>
-#include <glib.h>
-#include <string.h>
-#include <ctype.h>
-#include <sys/time.h>
-#include <time.h>
-#include <unistd.h>
-#include <sys/types.h>
-#include <stdint.h>
-#include <zlib.h>
-#include <fnmatch.h>
-
-/*
- We assume the syslog files belong to one host, i.e. we do not
- consider the hostname at all
-*/
-
-/*
- REGEX: we do not use posix regular expressions (libc implementation
- is too slow). fnmatch() is also slow.
- Future versions may use our own implementation of a DFA. But currently we
- just use strstr (no regex support at all)
-*/
-
-//#define STRMATCH(pattern, string) (fnmatch (pattern, string, FNM_CASEFOLD) == 0)
-#define STRMATCH(pattern, string) (strcasestr (string, pattern) != NULL)
-
-
-//#define EPOOL_BLOCK_SIZE 512
-//#define EPOOL_MAX_SIZE 128
-#define EPOOL_BLOCK_SIZE 2048
-#define EPOOL_MAX_SIZE 128
-#define MAX_LOGFILES 32
-//#define EPOOL_DEBUG
-//#define DEBUG
-
-typedef struct _SList SList;
-struct _SList {
- gpointer data;
- SList *next;
-};
-
-typedef struct _NQList NQList;
-struct _NQList {
- char *from;
- char *to;
- char dstatus;
- time_t ltime;
- NQList *next;
-};
-
-typedef struct _TOList TOList;
-struct _TOList {
- char *to;
- char *relay;
- char dstatus;
- time_t ltime;
- TOList *next;
-};
-
-#define MatchTypeQID 1
-#define MatchTypeRelLineNr 2
-
-typedef struct _MatchList MatchList;
-struct _MatchList {
- unsigned int mtype;
- char *id;
- time_t ltime;
- unsigned long rel_line_nr;
- MatchList *next;
-};
-
-#ifdef DEBUG
- GHashTable *smtpd_debug_alloc;
- GHashTable *qmgr_debug_alloc;
- GHashTable *filter_debug_alloc;
- GHashTable *smtpd_debug_free;
- GHashTable *qmgr_debug_free;
- GHashTable *filter_debug_free;
-#endif
-
-// EPool: Entry related memory pools
-
-
-#ifdef EPOOL_DEBUG
-int ep_allocated;
-int ep_maxalloc;
-#endif
-
-typedef struct _EPool EPool;
-struct _EPool {
- SList *blocks; // allocated usiing g_slice_alloc(EPOOL_BLOCK_SIZE)
- SList *mblocks; // allocated use g_malloc
- int cpos;
-#ifdef EPOOL_DEBUG
- int allocated;
-#endif
-};
-
-typedef struct {
- EPool ep;
- GHashTable *smtpd_h;
- GHashTable *qmgr_h;
- GHashTable *filter_h;
- //GHashTable *track_h;
- gzFile stream[MAX_LOGFILES];
- char *from;
- char *to;
- time_t year[MAX_LOGFILES];
- time_t start;
- time_t end;
- time_t ctime;
- MatchList *match_list;
- char *server;
- char *msgid;
- char *strmatch;
- unsigned long limit;
- unsigned long count;
- int verbose;
- unsigned int exclude_greylist:1;
- unsigned int exclude_ndrs:1;
-} LParser;
-
-typedef struct _SEntry SEntry;
-typedef struct _QEntry QEntry;
-typedef struct _FEntry FEntry;
-
-typedef struct _LogEntry LogEntry;
-typedef struct _LogList LogList;
-
-struct _LogEntry {
- const char *text;
- unsigned long linenr;
- LogEntry *next;
-};
-
-struct _LogList {
- LogEntry *log;
- LogEntry *logs_last; // pointer to last log (speedup add log)
-};
-
-// SEntry: Store SMTPD related logs
-
-struct _SEntry {
-
- EPool ep;
- LogList loglist;
-
- int pid;
-
- SList *refs;
-
- NQList *nqlist;
-
- char *connect;
-
- // time,rel_line_nr is used as cursor/ID
- time_t ltime;
- unsigned long rel_line_nr;
-
- //unsigned int external:1; // not from local host
- unsigned int disconnect:1;
- unsigned int strmatch:1;
-
-};
-
-// QEntry: Store Queue (qmgr, smtp, lmtp) related logs
-
-struct _QEntry {
-
- EPool ep;
- LogList loglist;
-
- char *qid;
-
- SEntry *smtpd;
- FEntry *filter;
-
- TOList *tolist;
-
- unsigned int size;
- char *from;
- char *client;
- char *msgid;
-
- unsigned int cleanup:1;
- unsigned int removed:1;
- unsigned int filtered:1; // set when passed via lmtp to filter
- unsigned int strmatch:1;
-};
-
-// FEntry: Store filter (proxprox) related logs
-
-struct _FEntry {
-
- EPool ep;
- LogList loglist;
-
- char *logid; // proxprox log id
-
- TOList *tolist;
-
- float ptime;
-
- unsigned int finished:1;
- unsigned int strmatch:1;
-};
-
-// Prototypes
-void debug_error (char *msg, const char *line);
-
-EPool *epool_init (EPool *ep);
-void epool_free (EPool *ep);
-gpointer epool_alloc (EPool *ep, int size);
-char *epool_strndup (EPool *ep, const char *s, int len);
-char *epool_strndup0 (EPool *ep, const char *s, int len);
-char *epool_strdup (EPool *ep, const char *s);
-
-void loglist_print (LogList *loglist);
-void loglist_add (EPool *ep, LogList *loglist, const char *text, int len, unsigned long linenr);
-
-SEntry *sentry_new (int pid, time_t ltime, unsigned long rel_line_nr);
-SEntry *sentry_get (LParser *parser, int pid, time_t ltime, unsigned long rel_line_nr);
-void sentry_ref_add (SEntry *sentry, QEntry *qentry);
-int sentry_ref_del (SEntry *sentry, QEntry *qentry);
-void sentry_ref_finalize (LParser *parser, SEntry *sentry);
-int sentry_ref_rem_unneeded (LParser *parser, SEntry *sentry);
-void sentry_nqlist_add (SEntry *sentry, time_t ltime, const char *from, int from_len,
- const char *to, int to_len, char dstatus);
-void sentry_print (LParser *parser, SEntry *sentry);
-void sentry_set_connect (SEntry *sentry, const char *connect, int len);
-void sentry_free_noremove (SEntry *sentry);
-void sentry_free (LParser *parser, SEntry *sentry);
-void sentry_cleanup_hash (gpointer key, gpointer value, gpointer user_data);
-
-
-QEntry *qentry_new (const char *qid);
-QEntry *qentry_get (LParser *parser, const char *qid);
-void qentry_tolist_add (QEntry *qentry, time_t ltime, char dstatus, const char *to,
- int to_len, const char *relay, int relay_len);
-void qentry_set_from (QEntry *qentry, const char *from, int len);
-void qentry_set_msgid (QEntry *qentry, const char *msgid, int len);
-void qentry_set_client (QEntry *qentry, const char *client, int len);
-void qentry_print (LParser *parser, QEntry *qentry);
-void qentry_finalize (LParser *parser, QEntry *qentry);
-void qentry_free_noremove (QEntry *qentry);
-void qentry_free (LParser *parser, QEntry *qentry);
-void qentry_cleanup_hash (gpointer key, gpointer value, gpointer user_data);
-
-
-FEntry *fentry_new (const char *logid);
-FEntry *fentry_get (LParser *parser, const char *logid);
-void fentry_tolist_add (FEntry *fentry, char dstatus, const char *to,
- int to_len, const char *qid, int qid_len);
-void fentry_free_noremove (FEntry *fentry);
-void fentry_free (LParser *parser, FEntry *fentry);
-void fentry_cleanup_hash (gpointer key, gpointer value, gpointer user_data);
-
-LParser *parser_new ();
-void parser_free (LParser *parser);
-
-//char *parser_track (LParser *parser, const char *qid, gboolean insert);
-
-// Implementations
-
-// Checksum Macros
-#define PROXPROX 0xE0E4DEF0
-#define PMG_SMTP_FILTER 0x0A85A6B7
-#define POSTFIX_POSTSCREEN 0xD17E2019
-#define POSTFIX_QMGR 0x48465316
-#define POSTFIX_SMTP 0x4A466014
-#define POSTFIX_LMTP 0x43466014
-#define POSTFIX_LOCAL 0x484F05AF
-#define POSTFIX_ERROR 0x4B5E13AE
-#define POSTFIX_SMTPD 0x466014AE
-#define POSTFIX_CLEANUP 0x05A8BAC1
-
-//#define LOGPATH "./log/"
-#define LOGPATH "/var/log/"
-//#define LOGPATH "/root/testlog/"
-
-static const char *logfiles[] = {
- LOGPATH "syslog",
- LOGPATH "syslog.1",
- LOGPATH "syslog.2.gz",
- LOGPATH "syslog.3.gz",
- LOGPATH "syslog.4.gz",
- LOGPATH "syslog.5.gz",
- LOGPATH "syslog.6.gz",
- LOGPATH "syslog.7.gz",
- LOGPATH "syslog.8.gz",
- LOGPATH "syslog.9.gz",
- LOGPATH "syslog.10.gz",
- LOGPATH "syslog.11.gz",
- LOGPATH "syslog.12.gz",
- LOGPATH "syslog.13.gz",
- LOGPATH "syslog.14.gz",
- LOGPATH "syslog.15.gz",
- LOGPATH "syslog.16.gz",
- LOGPATH "syslog.17.gz",
- LOGPATH "syslog.18.gz",
- LOGPATH "syslog.19.gz",
- LOGPATH "syslog.20.gz",
- LOGPATH "syslog.21.gz",
- LOGPATH "syslog.22.gz",
- LOGPATH "syslog.23.gz",
- LOGPATH "syslog.24.gz",
- LOGPATH "syslog.25.gz",
- LOGPATH "syslog.26.gz",
- LOGPATH "syslog.27.gz",
- LOGPATH "syslog.28.gz",
- LOGPATH "syslog.29.gz",
- LOGPATH "syslog.30.gz",
- LOGPATH "syslog.31.gz",
-};
-
-void
-debug_error (char *msg, const char *line)
-{
-#ifdef DEBUG
- fprintf (stderr, "ERROR: %s\n", msg);
- if (line) fprintf (stderr, "LINE: %s\n", line);
-
- G_BREAKPOINT();
-
- exit (-1);
-#endif
-}
-
-EPool *
-epool_init (EPool *ep)
-{
- gpointer data;
- SList *blocks;
-
- data = g_slice_alloc0(EPOOL_BLOCK_SIZE);
- blocks = (SList *)data;
-
-#ifdef EPOOL_DEBUG
- ep->allocated += EPOOL_BLOCK_SIZE;
- ep_allocated += EPOOL_BLOCK_SIZE;
- ep_maxalloc = (ep_allocated > ep_maxalloc) ? ep_allocated : ep_maxalloc;
-#endif
-
-
- blocks->data = data;
- blocks->next = NULL;
-
- ep->blocks = blocks;
- ep->cpos = sizeof (SList);
-
- return ep;
-}
-
-void
-epool_free (EPool *ep)
-{
- SList *l;
- gpointer data;
-
-#ifdef EPOOL_DEBUG
- ep_allocated -= ep->allocated;
- printf ("MEM: %d\n", ep_allocated);
-#endif
-
-#ifdef DEBUG
- return;
-#endif
-
- l = ep->mblocks;
- while (l) {
- data = l->data;
- l = l->next;
- g_free (data);
- }
-
- l = ep->blocks;
- while (l) {
- data = l->data;
- l = l->next;
- g_slice_free1(EPOOL_BLOCK_SIZE, data);
- }
-}
-
-gpointer
-epool_alloc (EPool *ep, int size)
-{
- int rs = (size + 3) & ~3;
- int space = EPOOL_BLOCK_SIZE - sizeof (SList) - ep->cpos;
- gpointer data;
-
- if (size > EPOOL_MAX_SIZE) {
- SList *blocks;
- if (space >= sizeof (SList)) {
- blocks = (SList *)((char *)ep->blocks->data + ep->cpos);
- ep->cpos += sizeof (SList);
- } else {
- blocks = (SList *)epool_alloc (ep, sizeof (SList));
- }
-
- data = g_malloc (size);
-#ifdef EPOOL_DEBUG
- ep->allocated += size;
- ep_allocated += size;
- ep_maxalloc = (ep_allocated > ep_maxalloc) ? ep_allocated : ep_maxalloc;
-#endif
- blocks->data = data;
- blocks->next = ep->mblocks;
-
- ep->mblocks = blocks;
-
- return data;
-
- } else if (space >= rs) {
- data = (char *)ep->blocks->data + ep->cpos;
- ep->cpos += rs;
-
- return data;
-
- } else {
- SList *blocks = (SList *)((char *)ep->blocks->data + ep->cpos);
-
- data = g_slice_alloc0 (EPOOL_BLOCK_SIZE);
- blocks->data = data;
- blocks->next = ep->blocks;
-
- ep->blocks = blocks;
- ep->cpos = rs;
-
-#ifdef EPOOL_DEBUG
- ep->allocated += EPOOL_BLOCK_SIZE;
- ep_allocated += EPOOL_BLOCK_SIZE;
- ep_maxalloc = (ep_allocated > ep_maxalloc) ? ep_allocated : ep_maxalloc;
-#endif
-
- return data;
- }
-}
-
-char *
-epool_strndup (EPool *ep, const char *s, int len)
-{
- int l = len + 1;
- char *res = epool_alloc (ep, l);
- strncpy (res, s, l);
- return res;
-}
-
-char *
-epool_strndup0 (EPool *ep, const char *s, int len)
-{
- char *res = epool_alloc (ep, len + 1);
- strncpy (res, s, len);
- res[len] = 0;
-
- return res;
-}
-
-char *
-epool_strdup (EPool *ep, const char *s)
-{
- int l = strlen (s) + 1;
- char *res = epool_alloc (ep, l);
- strncpy (res, s, l);
- return res;
-}
-
-void
-loglist_print (LogList *loglist)
-{
- LogEntry *log = loglist->log;
- while (log) {
- printf ("L%08lX %s", log->linenr, log->text);
- log = log->next;
- }
-}
-
-
-void
-loglist_add (EPool *ep, LogList *loglist, const char *text, int len, unsigned long linenr)
-{
- LogEntry *log;
-
-#ifdef DEBUG
- if (len != strlen (text)) {
- debug_error ("string with wrong len", NULL);
- }
-#endif
- if (text[len] != 0) {
- debug_error ("string is not null terminated", NULL);
- return;
- }
-
- log = epool_alloc (ep, sizeof (LogEntry));
-
- log->text = epool_strndup (ep, text, len);
- log->linenr = linenr;
- log->next = NULL;
-
- if (loglist->logs_last) {
- loglist->logs_last->next = log;
- loglist->logs_last = log;
- } else {
- loglist->log = log;
- loglist->logs_last = log;
- }
-
- return;
-}
-
-SEntry*
-sentry_new (int pid, time_t ltime, unsigned long rel_line_nr)
-{
- SEntry *sentry;
- SList *blocks;
- int cpos;
-
- sentry = (SEntry *)g_slice_alloc0(EPOOL_BLOCK_SIZE);
- sentry->pid = pid;
- sentry->ltime = ltime;
- sentry->rel_line_nr = rel_line_nr;
-
-#ifdef EPOOL_DEBUG
- sentry->ep.allocated += EPOOL_BLOCK_SIZE;
- ep_allocated += EPOOL_BLOCK_SIZE;
- ep_maxalloc = (ep_allocated > ep_maxalloc) ? ep_allocated : ep_maxalloc;
-#endif
-
-#ifdef DEBUG
- {
- SEntry *se;
- if ((se = g_hash_table_lookup (smtpd_debug_alloc, sentry))) {
- debug_error ("SEntry already alloced", NULL);
- } else {
- g_hash_table_insert (smtpd_debug_alloc, sentry, sentry);
- }
- }
-#endif
-
- cpos = sizeof (SEntry);
-
- blocks = (SList *)((char *)sentry + cpos);
-
- cpos += sizeof (SList);
-
- blocks->data = sentry;
- blocks->next = NULL;
-
- sentry->ep.blocks = blocks;
- sentry->ep.cpos = cpos;
-
- return sentry;
-}
-
-SEntry *
-sentry_get (LParser *parser, int pid, time_t ltime, unsigned long rel_line_nr)
-{
- SEntry *sentry;
-
- if ((sentry = g_hash_table_lookup (parser->smtpd_h, &pid))) {
- return sentry;
- } else {
-
- if ((sentry = sentry_new (pid, ltime, rel_line_nr))) {
- g_hash_table_insert (parser->smtpd_h, &sentry->pid, sentry);
- }
-
- return sentry;
- }
-}
-
-void
-sentry_ref_add (SEntry *sentry, QEntry *qentry)
-{
- SList *l;
-
- if (qentry->smtpd) {
- if (qentry->smtpd != sentry) {
- debug_error ("qentry ref already set", NULL);
- }
- return;
- }
-
- l = sentry->refs;
- while (l) {
- if (l->data == qentry) return;
- l = l->next;
- }
-
- l = epool_alloc (&sentry->ep, sizeof (SList));
-
- qentry->smtpd = sentry;
-
- l->data = qentry;
- l->next = sentry->refs;
-
- sentry->refs = l;
-}
-
-int
-sentry_ref_del (SEntry *sentry, QEntry *qentry)
-{
- SList *l = sentry->refs;
- int count = 0;
-
- if (!qentry->smtpd) {
- debug_error ("qentry does not hav a qentry ref", NULL);
- return 0;
- }
-
- l = sentry->refs;
-
- while (l) {
- QEntry *qe = (QEntry *)l->data;
- if (qe == qentry) {
- l->data = NULL;
- } else {
- if (qe && qe->cleanup) count++;
- }
- l = l->next;
- }
-
- return count;
-}
-
-void
-sentry_ref_finalize (LParser *parser, SEntry *sentry)
-{
- SList *l = sentry->refs;
-
- int count = 0;
-
- while (l) {
- SList *cl = l;
- QEntry *qe = l->data;
-
- l = l->next;
-
- if (!qe) continue;
-
- count++;
-
- if (!qe->removed) continue;
-
- FEntry *fe = qe->filter;
-
- if (fe && !fe->finished) continue;
-
- count--;
-
- qentry_print (parser, qe);
-
- cl->data = NULL;
- qe->smtpd = NULL;
-
- qentry_free (parser, qe);
-
- if (fe) fentry_free (parser, fe);
-
- }
-
- if (!count) sentry_free_noremove (sentry);
-}
-
-int
-sentry_ref_rem_unneeded (LParser *parser, SEntry *sentry)
-{
- SList *l = sentry->refs;
- int count = 0;
-
- while (l) {
- QEntry *qe = (QEntry *)l->data;
-
- if (qe) {
- if (!qe->cleanup) {
- qe->smtpd = NULL;
- qentry_free (parser, qe);
- l->data = NULL;
- } else {
- count++;
- }
- }
-
- l = l->next;
- }
-
- return count;
-}
-
-void
-sentry_nqlist_add (SEntry *sentry, time_t ltime, const char *from, int from_len,
- const char *to, int to_len, char dstatus)
-{
- NQList *nq = (NQList *)epool_alloc (&sentry->ep, sizeof (NQList));
-
- nq->from = epool_strndup0 (&sentry->ep, from, from_len);
- nq->to = epool_strndup0 (&sentry->ep, to, to_len);
- nq->dstatus = dstatus;
-
- nq->next = sentry->nqlist;
- nq->ltime = ltime;
- sentry->nqlist = nq;
-}
-
-void
-sentry_print (LParser *parser, SEntry *sentry)
-{
- NQList *nq;
-
- if (parser->msgid) return;
-
- if (parser->server) {
- if (!sentry->connect) return;
- if (!strcasestr (sentry->connect, parser->server)) return;
- }
-
- MatchList *match = parser->match_list;
- if (match) {
- int found = 0;
- while(match) {
- if (match->mtype == MatchTypeQID) {
- return;
- } else if (match->mtype == MatchTypeRelLineNr) {
- if (match->ltime == sentry->ltime && match->rel_line_nr == sentry->rel_line_nr) {
- found = 1;
- break;
- }
- } else {
- g_error("implement me");
- }
- match = match->next;
- }
- if (!found) return;
- }
-
- if (parser->from || parser->to ||
- parser->exclude_greylist || parser->exclude_ndrs) {
- nq = sentry->nqlist;
- int found = 0;
- while (nq) {
- if (parser->from) {
- if (!*(parser->from)) {
- if (*(nq->from)) nq->dstatus = 0;
- } else if (!STRMATCH(parser->from, nq->from)) {
- nq->dstatus = 0;
- }
- }
-
- if (parser->exclude_greylist && nq->dstatus == 'G') nq->dstatus = 0;
-
- if (parser->exclude_ndrs && nq->from && !*nq->from) nq->dstatus = 0;
-
- if (parser->to && nq->to && !STRMATCH(parser->to, nq->to)) {
- nq->dstatus = 0;
- }
-
- if (nq->dstatus != 0) found = 1;
-
- nq = nq->next;
- }
- if (!found) return;
- }
-
- if (parser->strmatch && !sentry->strmatch) return;
-
- if (parser->verbose) {
-
- printf ("SMTPD: T%08lXL%08lX\n", sentry->ltime, sentry->rel_line_nr);
-
- printf ("CTIME: %08lX\n", parser->ctime);
-
- if (sentry->connect) { printf ("CLIENT: %s\n", sentry->connect); }
- //printf ("EXTERNAL: %d\n", sentry->external);
-
- }
-
- nq = sentry->nqlist;
- while (nq) {
- if (nq->from && nq->to && nq->dstatus) {
- printf ("TO:%08lX:T%08lXL%08lX:%c: from <%s> to <%s>\n", nq->ltime,
- sentry->ltime, sentry->rel_line_nr, nq->dstatus,
- nq->from, nq->to);
- parser->count++;
- }
- nq = nq->next;
- }
-
- if (!parser->verbose) { fflush (stdout); return; }
-
- if (parser->verbose > 1) {
- printf ("LOGS:\n");
- loglist_print (&sentry->loglist);
- }
-
- printf ("\n");
-
- fflush (stdout);
-}
-
-void
-sentry_set_connect (SEntry *sentry, const char *connect, int len)
-{
- if (sentry->connect) {
-#ifdef DEBUG
- if (strncmp (sentry->connect, connect, len)) {
- debug_error ("duplicate connect", NULL);
- }
-#endif
- } else {
- sentry->connect = epool_strndup0 (&sentry->ep, connect, len);
- }
-}
-
-void
-sentry_free_noremove (SEntry *sentry)
-{
- SList *l;
- gpointer data;
-
-#ifdef EPOOL_DEBUG
- ep_allocated -= sentry->ep.allocated;
- printf ("MEM: %d\n", ep_allocated);
-#endif
-
-#ifdef DEBUG
- {
- SEntry *se;
- if ((se = g_hash_table_lookup (smtpd_debug_free, sentry))) {
- debug_error ("SEntry already freed", NULL);
- } else {
- g_hash_table_insert (smtpd_debug_free, sentry, sentry);
- }
- }
- return;
-#endif
-
- l = sentry->ep.mblocks;
- while (l) {
- data = l->data;
- l = l->next;
- g_free (data);
- }
-
- l = sentry->ep.blocks;
- while (l) {
- data = l->data;
- l = l->next;
- g_slice_free1(EPOOL_BLOCK_SIZE, data);
- }
-}
-
-void
-sentry_free (LParser *parser, SEntry *sentry)
-{
- g_hash_table_remove (parser->smtpd_h, &sentry->pid);
-
- sentry_free_noremove (sentry);
-}
-
-void
-sentry_cleanup_hash (gpointer key,
- gpointer value,
- gpointer user_data)
-{
- SEntry *se = value;
- LParser *parser = (LParser *)user_data;
-
- sentry_print (parser, se);
- sentry_free_noremove (se);
-}
-
-#ifdef DEBUG
-void
-sentry_debug_alloc (gpointer key,
- gpointer value,
- gpointer user_data)
-{
- LParser *parser = (LParser *)user_data;
- SEntry *se = value;
- SEntry *fe;
-
- if ((fe = g_hash_table_lookup (smtpd_debug_free, se))) {
- return;
- }
-
- printf ("FOUND ALLOCATED SENTRY:\n");
- sentry_print (parser, se);
-
- exit (-1);
-}
-#endif
-
-// QEntry
-
-QEntry*
-qentry_new (const char *qid)
-{
- QEntry *qentry;
- SList *blocks;
- int cpos;
- char *qid_cp;
-
- qentry = (QEntry *)g_slice_alloc0(EPOOL_BLOCK_SIZE);
-
-#ifdef EPOOL_DEBUG
- qentry->ep.allocated += EPOOL_BLOCK_SIZE;
- ep_allocated += EPOOL_BLOCK_SIZE;
- ep_maxalloc = (ep_allocated > ep_maxalloc) ? ep_allocated : ep_maxalloc;
-#endif
-
-#ifdef DEBUG
- {
- QEntry *qe;
- if ((qe = g_hash_table_lookup (qmgr_debug_alloc, qentry))) {
- debug_error ("QEntry already alloced", NULL);
- } else {
- g_hash_table_insert (qmgr_debug_alloc, qentry, qentry);
- }
- }
-#endif
-
- cpos = sizeof (QEntry);
-
- blocks = (SList *)((char *)qentry + cpos);
-
- cpos += sizeof (SList);
-
- blocks->data = qentry;
- blocks->next = NULL;
-
- qentry->qid = qid_cp = (char *)qentry + cpos;
- while ((*qid_cp++ = *qid++)) cpos++;
-
- cpos = (cpos + 4) & ~3;
-
- qentry->ep.blocks = blocks;
- qentry->ep.cpos = cpos;;
-
- return qentry;
-}
-
-void
-qentry_tolist_add (QEntry *qentry, time_t ltime, char dstatus, const char *to, int to_len,
- const char *relay, int relay_len)
-{
- TOList *tl = (TOList *)epool_alloc (&qentry->ep, sizeof (TOList));
-
- tl->to = epool_strndup0 (&qentry->ep, to, to_len);
- tl->relay = epool_strndup0 (&qentry->ep, relay, relay_len);
- tl->dstatus = dstatus;
- tl->ltime = ltime;
- tl->next = qentry->tolist;
-
- qentry->tolist = tl;
-}
-
-void
-qentry_set_from (QEntry *qentry, const char *from, int len)
-{
- if (qentry->from) {
-#ifdef DEBUG
- if (strncmp (qentry->from, from, len)) {
- debug_error ("duplicate from", NULL);
- }
-#endif
- } else {
- qentry->from = epool_strndup0 (&qentry->ep, from, len);
- }
-}
-
-void
-qentry_set_msgid (QEntry *qentry, const char *msgid, int len)
-{
- if (qentry->msgid) {
-#ifdef DEBUG
- if (strncmp (qentry->msgid, msgid, len)) {
- debug_error ("duplicate msgid", NULL);
- }
-#endif
- } else {
- qentry->msgid = epool_strndup0 (&qentry->ep, msgid, len);
- }
-}
-
-void
-qentry_set_client (QEntry *qentry, const char *client, int len)
-{
- if (qentry->client) {
-#ifdef DEBUG
- if (strncmp (qentry->client, client, len)) {
- debug_error ("duplicate client", NULL);
- }
-#endif
- } else {
- qentry->client = epool_strndup0 (&qentry->ep, client, len);
- }
-}
-
-void
-qentry_print (LParser *parser, QEntry *qentry)
-{
- TOList *tl, *fl;
- SEntry *se = qentry->smtpd;
- FEntry *fe = qentry->filter;
-
- if (parser->msgid) {
- if (!qentry->msgid) return;
- if (strcasecmp (parser->msgid, qentry->msgid)) return;
- }
-
- MatchList *match = parser->match_list;
- if (match) {
- int found = 0;
- while(match) {
- if (match->mtype == MatchTypeQID) {
- if ((fe && !strcmp (fe->logid, match->id)) ||
- (!strcmp (qentry->qid, match->id))) {
- found = 1;
- break;
- }
- } else if (match->mtype == MatchTypeRelLineNr) {
- if (se && match->ltime == se->ltime && match->rel_line_nr == se->rel_line_nr) {
- found = 1;
- break;
- }
- } else {
- g_error("implement me");
- }
- match = match->next;
- }
- if (!found) return;
- }
-
- if (parser->server) {
- int found = 0;
- if (se && se->connect && strcasestr (se->connect, parser->server)) found = 1;
- if (qentry->client && strcasestr (qentry->client, parser->server)) found = 1;
-
- if (!found) return;
- }
-
- if (parser->from) {
- if (!qentry->from) return;
- if (!*(parser->from)) {
- if (*(qentry->from)) return;
- } else if (!STRMATCH(parser->from, qentry->from)) {
- return;
- }
- } else {
- if (parser->exclude_ndrs && qentry->from && !*qentry->from) return;
- }
-
- if (parser->to) {
- tl = qentry->tolist;
- int found = 0;
- while (tl) {
- if (parser->to && !STRMATCH(parser->to, tl->to)) {
- tl->to = NULL;
- } else {
- found = 1;
- }
- tl = tl->next;
- }
- if (!found) return;
- }
-
- if (parser->strmatch &&
- !(qentry->strmatch || (se && se->strmatch) || (fe && fe->strmatch)))
- return;
-
-
- if (parser->verbose) {
-
- printf ("QENTRY: %s\n", qentry->qid);
-
- printf ("CTIME: %08lX\n", parser->ctime);
- printf ("SIZE: %u\n", qentry->size);
-
- if (qentry->client) {
- printf ("CLIENT: %s\n", qentry->client);
- } else if (se && se->connect) {
- printf ("CLIENT: %s\n", se->connect);
- }
-
- if (qentry->msgid) { printf ("MSGID: %s\n", qentry->msgid); }
-
- }
-
- tl = qentry->tolist;
- while (tl) {
- if (tl->to) {
- fl = NULL;
- if (fe && tl->dstatus == '2') {
- fl = fe->tolist;
- while (fl) {
- if (fl->to && !strcmp (tl->to, fl->to)) {
- break;
- }
- fl = fl->next;
- }
- }
- char *to;
- char dstatus;
- char *relay;
-
- if (fl) {
- to = fl->to;
- dstatus = fl->dstatus;
- relay = fl->relay;
- } else {
- to = tl->to;
- dstatus = tl->dstatus;
- relay = tl->relay;
- }
-
- printf ("TO:%08lX:%s:%c: from <%s> to <%s> (%s)\n", tl->ltime, qentry->qid, dstatus, qentry->from ? qentry->from : "", to ? to : "", relay ? relay : "none");
-
- parser->count++;
- }
- tl = tl->next;
- }
-
-
- if (!parser->verbose) { fflush (stdout); return; }
-
- if (parser->verbose > 1) {
-
- if (se && se->loglist.log) {
- printf ("SMTP:\n");
- loglist_print (&se->loglist);
- }
-
- if (fe && fe->loglist.log) {
- printf ("FILTER: %s\n", fe->logid);
- loglist_print (&fe->loglist);
- }
-
- if (qentry->loglist.log) {
- printf ("QMGR:\n");
- loglist_print (&qentry->loglist);
- }
- }
-
- printf ("\n");
-
- fflush (stdout);
-
- //sleep (1);
-}
-
-QEntry *
-qentry_get (LParser *parser, const char *qid)
-{
- QEntry *qentry;
-
- if ((qentry = g_hash_table_lookup (parser->qmgr_h, qid))) {
- return qentry;
- } else {
- if ((qentry = qentry_new (qid))) {
- g_hash_table_insert (parser->qmgr_h, qentry->qid, qentry);
- }
-
- return qentry;
- }
-}
-
-void
-qentry_free_noremove (QEntry *qentry)
-{
- SList *l;
- gpointer data;
- SEntry *se;
-
- if ((se = qentry->smtpd)) {
- if (sentry_ref_del (se, qentry) == 0) {
- if (se->disconnect) {
- sentry_free_noremove (se);
- }
- }
- }
-
-#ifdef EPOOL_DEBUG
- ep_allocated -= qentry->ep.allocated;
- printf ("MEM: %d\n", ep_allocated);
-#endif
-
-#ifdef DEBUG
- {
- QEntry *qe;
- if ((qe = g_hash_table_lookup (qmgr_debug_free, qentry))) {
- debug_error ("QEntry already freed", NULL);
- } else {
- g_hash_table_insert (qmgr_debug_free, qentry, qentry);
- }
- }
- return;
-#endif
-
- l = qentry->ep.mblocks;
- while (l) {
- data = l->data;
- l = l->next;
- g_free (data);
- }
-
- l = qentry->ep.blocks;
- while (l) {
- data = l->data;
- l = l->next;
- g_slice_free1(EPOOL_BLOCK_SIZE, data);
- }
-}
-
-void
-qentry_free (LParser *parser, QEntry *qentry)
-{
- g_hash_table_remove (parser->qmgr_h, qentry->qid);
-
- qentry_free_noremove (qentry);
-}
-
-void
-qentry_cleanup_hash (gpointer key,
- gpointer value,
- gpointer user_data)
-{
- QEntry *qe = value;
- LParser *parser = (LParser *)user_data;
-
- qentry_print (parser, qe);
- qentry_free_noremove (qe);
-}
-
-void
-qentry_finalize (LParser *parser, QEntry *qentry)
-{
- if (qentry && qentry->removed) {
- SEntry *se = qentry->smtpd;
-
- if (se && !se->disconnect) return;
-
- FEntry *fe = qentry->filter;
-
- if (fe && !fe->finished) return;
-
- qentry_print (parser, qentry);
- qentry_free (parser, qentry);
-
- if (fe) fentry_free (parser, fe);
- }
-}
-
-// FEntry
-
-FEntry*
-fentry_new (const char *logid)
-{
- FEntry *fentry;
- SList *blocks;
- int cpos;
- char *logid_cp;
-
- fentry = (FEntry *)g_slice_alloc0(EPOOL_BLOCK_SIZE);
-
-#ifdef EPOOL_DEBUG
- fentry->ep.allocated += EPOOL_BLOCK_SIZE;
- ep_allocated += EPOOL_BLOCK_SIZE;
- ep_maxalloc = (ep_allocated > ep_maxalloc) ? ep_allocated : ep_maxalloc;
-#endif
-
-#ifdef DEBUG
- {
- FEntry *fe;
- if ((fe = g_hash_table_lookup (filter_debug_alloc, fentry))) {
- debug_error ("FEntry already alloced", NULL);
- } else {
- g_hash_table_insert (filter_debug_alloc, fentry, fentry);
- }
- }
-#endif
-
- cpos = sizeof (FEntry);
-
- blocks = (SList *)((char *)fentry + cpos);
-
- cpos += sizeof (SList);
-
- blocks->data = fentry;
- blocks->next = NULL;
-
- fentry->logid = logid_cp = (char *)fentry + cpos;
- while ((*logid_cp++ = *logid++)) cpos++;
- cpos = (cpos + 4) & ~3;
-
- fentry->ep.blocks = blocks;
- fentry->ep.cpos = cpos;;
-
- return fentry;
-}
-
-FEntry *
-fentry_get (LParser *parser, const char *logid)
-{
- FEntry *fentry;
-
- if ((fentry = g_hash_table_lookup (parser->filter_h, logid))) {
- return fentry;
- } else {
- if ((fentry = fentry_new (logid))) {
- g_hash_table_insert (parser->filter_h, fentry->logid, fentry);
- }
-
- return fentry;
- }
-}
-
-void
-fentry_tolist_add (FEntry *fentry, char dstatus, const char *to, int to_len,
- const char *qid, int qid_len)
-{
- TOList *tl = (TOList *)epool_alloc (&fentry->ep, sizeof (TOList));
-
- tl->to = epool_strndup0 (&fentry->ep, to, to_len);
-
- if (qid) {
- tl->relay = epool_strndup0 (&fentry->ep, qid, qid_len);
- } else {
- tl->relay = NULL;
- }
- tl->dstatus = dstatus;
- tl->next = fentry->tolist;
-
- fentry->tolist = tl;
-}
-
-void
-fentry_free_noremove (FEntry *fentry)
-{
- SList *l;
- gpointer data;
-
-#ifdef EPOOL_DEBUG
- ep_allocated -= fentry->ep.allocated;
- printf ("MEM: %d\n", ep_allocated);
-#endif
-
-#ifdef DEBUG
- {
- FEntry *fe;
- if ((fe = g_hash_table_lookup (filter_debug_free, fentry))) {
- debug_error ("FEntry already freed", NULL);
- } else {
- g_hash_table_insert (filter_debug_free, fentry, fentry);
- }
- }
- return;
-#endif
-
- l = fentry->ep.mblocks;
- while (l) {
- data = l->data;
- l = l->next;
- g_free (data);
- }
-
- l = fentry->ep.blocks;
- while (l) {
- data = l->data;
- l = l->next;
- g_slice_free1(EPOOL_BLOCK_SIZE, data);
- }
-}
-
-void
-fentry_free (LParser *parser, FEntry *fentry)
-{
- g_hash_table_remove (parser->filter_h, fentry->logid);
-
- fentry_free_noremove (fentry);
-}
-
-void
-fentry_cleanup_hash (gpointer key,
- gpointer value,
- gpointer user_data)
-{
- FEntry *fe = value;
-
- fentry_free_noremove (fe);
-}
-
-// Parser
-
-LParser*
-parser_new ()
-{
- LParser *parser = g_malloc0 (sizeof (LParser));
- struct timeval tv;
- struct tm *ltime;
- int i;
-
- epool_init (&parser->ep);
-
- if (!(parser->smtpd_h = g_hash_table_new (g_int_hash, g_int_equal))) {
- return NULL;
- }
-
- if (!(parser->qmgr_h = g_hash_table_new (g_str_hash, g_str_equal))) {
- return NULL;
- }
-
- if (!(parser->filter_h = g_hash_table_new (g_str_hash, g_str_equal))) {
- return NULL;
- }
-
- //if (!(parser->track_h = g_hash_table_new (g_str_hash, g_str_equal))) {
- //return NULL;
- //}
-
- for (i = 0; i < MAX_LOGFILES; i++) {
- gettimeofday(&tv, NULL);
- tv.tv_sec -= 3600*24*i;
- ltime = localtime (&tv.tv_sec);
- parser->year[i] = ltime->tm_year + 1900;
- }
-
- return parser;
-}
-
-void
-parser_free (LParser *parser)
-{
- int i;
-
- for (i = 0; i < MAX_LOGFILES; i++) {
- if (parser->stream[i]) gzclose (parser->stream[i]);
- }
-
- epool_free (&parser->ep);
-
- g_hash_table_destroy (parser->smtpd_h);
- g_hash_table_destroy (parser->qmgr_h);
- g_hash_table_destroy (parser->filter_h);
- //g_hash_table_destroy (parser->track_h);
-
- g_free (parser);
-}
-
-#if 0
-char *
-parser_track (LParser *parser, const char *qid, gboolean insert)
-{
- char *res;
-
- if ((res = g_hash_table_lookup (parser->track_h, qid))) {
- return res;
- } else {
- if (insert && (res = epool_strdup (&parser->ep, qid))) {
- g_hash_table_insert (parser->track_h, res, res);
- return res;
- }
- }
- return NULL;
-}
-#endif
-
-#define LINEBUFSIZE 8192
-static int cur_year;
-static int cur_month = 0;
-static int cal_mtod[12] = {0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334};
-
-static time_t
-mkgmtime (struct tm *tm)
-{
- time_t res;
-
- int year = tm->tm_year + 1900;
- int mon = tm->tm_mon;
-
- res = (year - 1970) * 365 + cal_mtod[mon];
-
- // leap year corrections (gregorian calendar)
- if (mon <= 1) year -= 1;
- res += (year - 1968) / 4;
- res -= (year - 1900) / 100;
- res += (year - 1600) / 400;
-
- res += tm->tm_mday - 1;
- res = res*24 + tm->tm_hour;
- res = res*60 + tm->tm_min;
- res = res*60 + tm->tm_sec;
-
- return res;
-}
-
-#define JAN (('J'<<16)|('a'<<8)|'n')
-#define FEB (('F'<<16)|('e'<<8)|'b')
-#define MAR (('M'<<16)|('a'<<8)|'r')
-#define APR (('A'<<16)|('p'<<8)|'r')
-#define MAY (('M'<<16)|('a'<<8)|'y')
-#define JUN (('J'<<16)|('u'<<8)|'n')
-#define JUL (('J'<<16)|('u'<<8)|'l')
-#define AUG (('A'<<16)|('u'<<8)|'g')
-#define SEP (('S'<<16)|('e'<<8)|'p')
-#define OCT (('O'<<16)|('c'<<8)|'t')
-#define NOV (('N'<<16)|('o'<<8)|'v')
-#define DEC (('D'<<16)|('e'<<8)|'c')
-
-time_t
-parse_time (const char **text, int len)
-{
- time_t ltime = 0;
-
- int year = cur_year;
-
- int mon = 0;
- int mday = 0;
- int hour = 0;
- int min = 0;
- int sec = 0;
- int found;
-
- const char *line = *text;
-
- if (len == (LINEBUFSIZE - 1)) {
- debug_error ("skipping long line data", line);
- return 0;
- }
-
- if (len < 15) {
- debug_error ("skipping short line data", line);
- return 0;
- }
-
- // parse month
- int csum = (line[0]<<16) + (line[1]<<8) + line[2];
-
- switch (csum) {
- case JAN: mon = 0; break;
- case FEB: mon = 1; break;
- case MAR: mon = 2; break;
- case APR: mon = 3; break;
- case MAY: mon = 4; break;
- case JUN: mon = 5; break;
- case JUL: mon = 6; break;
- case AUG: mon = 7; break;
- case SEP: mon = 8; break;
- case OCT: mon = 9; break;
- case NOV: mon = 10; break;
- case DEC: mon = 11; break;
- default:
- debug_error ("unable to parse month", line);
- return 0;
- }
-
- // year change heuristik
- if (cur_month == 11 && mon == 0) {
- year++;
- }
- if (mon > cur_month) cur_month = mon;
-
- ltime = (year - 1970)*365 + cal_mtod[mon];
-
- // leap year corrections (gregorian calendar)
- if (mon <= 1) year -= 1;
- ltime += (year - 1968) / 4;
- ltime -= (year - 1900) / 100;
- ltime += (year - 1600) / 400;
-
- const char *cpos = line + 3;
-
- found = 0; while (isspace (*cpos)) { cpos++; found = 1; }
- if (!found) {
- debug_error ("missing spaces after month", line);
- return 0;
- }
-
- found = 0; while (isdigit (*cpos)) { mday = mday*10 + *cpos - '0'; cpos++; found++; }
- if (found < 1 || found > 2) {
- debug_error ("unable to parse day of month", line);
- return 0;
- }
-
- ltime += mday - 1;
-
- found = 0; while (isspace (*cpos)) { cpos++; found++; }
- if (!found) {
- debug_error ("missing spaces after day of month", line);
- return 0;
- }
-
- found = 0; while (isdigit (*cpos)) { hour = hour*10 + *cpos - '0'; cpos++; found++; }
- if (found < 1 || found > 2) {
- debug_error ("unable to parse hour", line);
- return 0;
- }
-
- ltime *= 24;
- ltime += hour;
-
- if (*cpos != ':') {
- debug_error ("missing collon after hour", line);
- return 0;
- }
- cpos++;
-
- found = 0; while (isdigit (*cpos)) { min = min*10 + *cpos - '0'; cpos++; found++; }
- if (found < 1 || found > 2) {
- debug_error ("unable to parse minute", line);
- return 0;
- }
-
- ltime *= 60;
- ltime += min;
-
- if (*cpos != ':') {
- debug_error ("missing collon after minute", line);
- return 0;
- }
- cpos++;
-
- found = 0; while (isdigit (*cpos)) { sec = sec*10 + *cpos - '0'; cpos++; found++; }
- if (found < 1 || found > 2) {
- debug_error ("unable to parse second", line);
- return 0;
- }
-
- ltime *= 60;
- ltime += sec;
-
- found = 0; while (isspace (*cpos)) { cpos++; found = 1; }
- if (!found) {
- debug_error ("missing spaces after time", line);
- return 0;
- }
-
- *text = cpos;
-
- return ltime;
-}
-
-
-int
-parser_count_files (LParser *parser)
-{
- int i;
- time_t start = parser->start;
- char linebuf[LINEBUFSIZE];
- const char *line;
- gzFile stream;
-
- for (i = 0; i < (MAX_LOGFILES - 1); i++) {
- cur_year = parser->year[i];
- cur_month = 0;
-
- if ((stream = gzopen (logfiles[i], "r"))) {
- if ((line = gzgets (stream, linebuf, LINEBUFSIZE))) {
- if (parse_time (&line, strlen (line)) < start) {
- break;
- }
- } else {
- return i;
- }
- gzclose (stream);
- } else {
- return i;
- }
- }
-
- return i + 1;
-}
-
-static char *
-parse_qid (const char **text, char *out, char delim, int maxlen)
-{
- const char *idx;
- char *copy = out;
-
- int found = 0;
- idx = *text;
- while (isxdigit (*idx)) { *copy++ = *idx++; found++; if (found > maxlen) break; }
-
- if (found > 1 && found < maxlen &&
- ((delim && (*idx == delim)) || (!delim && isspace (*idx)))) {
- *copy = 0;
- idx++;
- while (isspace (*idx)) idx++;
- *text = idx;
- return out;
- }
- return NULL;
-}
-
-void
-print_usage (const char *name)
-{
- fprintf (stderr, "usage: %s [OPTIONS] [OUTPUTFILENAME]\n", name);
- fprintf (stderr, "\t-f SENDER mails from SENDER\n");
- fprintf (stderr, "\t-t RECIPIENT mails to RECIPIENT\n");
- fprintf (stderr, "\t-h Server Server IP or Hostname\n");
- fprintf (stderr, "\t-s START start time (YYYY-MM-DD HH:MM:SS)\n");
- fprintf (stderr, "\t or seconds since epoch\n");
- fprintf (stderr, "\t-e END end time (YYYY-MM-DD HH:MM:SS)\n");
- fprintf (stderr, "\t or seconds since epoch\n");
- fprintf (stderr, "\t-m MSGID message ID (exact match)\n");
- fprintf (stderr, "\t-q QID queue ID (exact match), can be\n");
- fprintf (stderr, "\t specified multiple times.\n");
- fprintf (stderr, "\t-x STRING search for strings\n");
- fprintf (stderr, "\t-l LIMIT print max limit entries\n");
- fprintf (stderr, "\t-g exclude greylist entries\n");
- fprintf (stderr, "\t-n exclude NDR entries\n");
- fprintf (stderr, "\t-v verbose output (no logs)\n");
- fprintf (stderr, "\t-vv verbose output with logs\n");
-}
-
-
-// gzgets is ways too slow, so we do it our own way
-
-static char
-mygzgetc (gzFile stream)
-{
- int br;
- static char readbuf[16384];
- static char *readend = readbuf + sizeof (readbuf);
- static char *readpos = readbuf + sizeof (readbuf);
-
- if (readpos < readend) return *readpos++;
-
- if ((br = gzread (stream, readbuf, sizeof (readbuf))) <= 0) {
- return -1;
- } else {
- readpos = readbuf;
- readend = readbuf + br;
-
- return *readpos++;
- }
-}
-
-static char *
-mygzgets (gzFile stream, char *line, int bufsize)
-{
- char c=0;
- char *cpos;
-
- cpos = line;
- while (--bufsize > 0 && (c = mygzgetc(stream)) != -1) {
- *cpos++ = c;
- if (c == '\n')
- break;
- }
- if (c == -1 && cpos == line)
- return NULL;
- *cpos++ = '\0';
- return line;
-}
-
-
-extern char *optarg;
-extern int optind, opterr, optopt;
-
-int
-main (int argc, char * const argv[])
-{
- char linebuf[LINEBUFSIZE];
- char *line;
- char *inputfile = NULL;
-
- const char *text;
- const char *idx1;
- const char *idx2;
- const char *cpos;
- int found = 0;
- uint32_t csum_prog;
- unsigned long lines = 0;
- unsigned long rel_line_nr = 0;
- char qidbuf[30];
- int i;
-
- struct tm *ltime;
- struct timeval tv;
- time_t ctime, next_ctime, start, end;
-
- LParser *parser;
- int opt;
-
-#ifdef DEBUG
- smtpd_debug_alloc = g_hash_table_new (g_direct_hash, g_direct_equal);
- qmgr_debug_alloc = g_hash_table_new (g_direct_hash, g_direct_equal);
- filter_debug_alloc = g_hash_table_new (g_direct_hash, g_direct_equal);
- smtpd_debug_free = g_hash_table_new (g_direct_hash, g_direct_equal);
- qmgr_debug_free = g_hash_table_new (g_direct_hash, g_direct_equal);
- filter_debug_free = g_hash_table_new (g_direct_hash, g_direct_equal);
-#endif
-
- if (!(parser = parser_new ())) {
- fprintf (stderr, "unable to alloc parser structure\n");
- exit (-1);
- }
-
- while ((opt = getopt (argc, argv, "f:t:s:e:h:m:q:x:i:l:vgn")) != -1) {
- if (opt == 'f') {
- parser->from = epool_strdup (&parser->ep, optarg);
- } else if (opt == 't') {
- parser->to = epool_strdup (&parser->ep, optarg);
- } else if (opt == 'v') {
- parser->verbose += 1;
- } else if (opt == 'g') {
- parser->exclude_greylist = 1;
- } else if (opt == 'n') {
- parser->exclude_ndrs = 1;
- } else if (opt == 'h') {
- parser->server = epool_strdup (&parser->ep, optarg);
- } else if (opt == 'm') {
- parser->msgid = epool_strdup (&parser->ep, optarg);
- } else if (opt == 'q') {
- time_t ltime;
- unsigned long rel_line_nr;
- MatchList *match = (MatchList *)epool_alloc(&parser->ep, sizeof(MatchList));
- if (sscanf(optarg, "T%08lXL%08lX", <ime, &rel_line_nr) == 2) {
- match->mtype = MatchTypeRelLineNr;
- match->ltime = ltime;
- match->rel_line_nr = rel_line_nr;
- match->next = parser->match_list;
- parser->match_list = match;
- } else {
- match->mtype = MatchTypeQID;
- match->id = epool_strdup(&parser->ep, optarg);
- match->next = parser->match_list;
- parser->match_list = match;
- }
- } else if (opt == 'x') {
- parser->strmatch = epool_strdup (&parser->ep, optarg);
- } else if (opt == 'i') {
- inputfile = optarg;
- } else if (opt == 'l') {
- char *l;
- parser->limit = strtoul (optarg, &l, 0);
- if (!*optarg || *l) {
- fprintf (stderr, "unable to parse limit '%s'\n", optarg);
- exit (-1);
- }
- } else if (opt == 's') {
- // use strptime to convert time
- struct tm tm;
- char *res;
- if ((!(res = strptime (optarg, "%F %T", &tm)) &&
- !(res = strptime (optarg, "%s", &tm))) || *res) {
- fprintf (stderr, "unable to parse start time\n");
- exit (-1);
- } else {
- parser->start = mkgmtime (&tm);
- }
- } else if (opt == 'e') {
- struct tm tm;
- char *res;
- if ((!(res = strptime (optarg, "%F %T", &tm)) &&
- !(res = strptime (optarg, "%s", &tm))) || *res) {
- fprintf (stderr, "unable to parse end time\n");
- exit (-1);
- } else {
- parser->end = mkgmtime (&tm);
- }
- } else {
- print_usage (argv[0]);
- exit (-1);
- }
- }
-
- if (optind < argc) {
-
- if ((argc - optind) > 1) {
- print_usage (argv[0]);
- exit (-1);
- }
-
- char *tmpfn = g_strdup_printf ("/tmp/.proxtrack-%08X.txt", getpid ());
-
- if ((stdout = freopen (tmpfn, "w", stdout)) == NULL) {
- perror ("unable to open output file");
- exit (-1);
- }
- if (rename (tmpfn, argv[optind]) != 0) {
- perror ("unable to open output file");
- unlink (tmpfn);
- exit (-1);
- }
- }
-
- // we use gmtime exerywhere to speedup things, this can cause problems
- // when daylight saving time changes
-
- gettimeofday(&tv, NULL);
- ltime = localtime (&tv.tv_sec);
-
- if (!parser->start) {
- ltime->tm_sec = 0;
- ltime->tm_min = 0;
- ltime->tm_hour = 0;
- parser->start = mkgmtime (ltime);
- }
-
- ltime = localtime (&tv.tv_sec);
-
- if (!parser->end) {
- parser->end = mkgmtime (ltime);
- }
-
- if (parser->end < parser->start) {
- fprintf (stderr, "end time before start time\n");
- exit (-1);
- }
-
- int filecount;
- if (inputfile) {
- filecount = 1;
- } else if ((filecount = parser_count_files (parser)) <= 0) {
- fprintf (stderr, "unable to access log files\n");
- exit (-1);
- }
-
- printf ("# LogReader: %d\n", getpid());
-
- printf ("# Query options\n");
- if (parser->from) printf ("# Sender: %s\n", parser->from);
- if (parser->to) printf ("# Recipient: %s\n", parser->to);
- if (parser->server) printf ("# Server: %s\n", parser->server);
- if (parser->msgid) printf ("# MsgID: %s\n", parser->msgid);
-
- MatchList *match = parser->match_list;
- while (match) {
- if (match->mtype == MatchTypeQID) {
- printf ("# QID: %s\n", match->id);
- } else if (match->mtype == MatchTypeRelLineNr) {
- printf ("# QID: T%08lXL%08lX\n", match->ltime, match->rel_line_nr);
- } else {
- g_error("internal error - unknown match type %d\n", match->mtype);
- }
- match = match->next;
- }
-
- if (parser->strmatch) printf ("# Match: %s\n", parser->strmatch);
-
- strftime (linebuf, 256, "%F %T", gmtime (&parser->start));
- printf ("# Start: %s (%lu)\n", linebuf, parser->start);
- strftime (linebuf, 256, "%F %T", gmtime (&parser->end));
- printf ("# END: %s (%lu)\n", linebuf, parser->end);
- printf ("# End Query Options\n\n");
-
- fflush (stdout);
-
- start = parser->start;
- end = parser->end;
- ctime = 0;
-
- for (i = filecount - 1; i >= 0; i--) {
- gpointer stream;
-
- // syslog files does not conain years, so we must compute then
- // cur_year is the assumed start year
-
- cur_month = 0;
- cur_year = parser->year[i];
-
- if (i <= 1) {
- if (inputfile && strlen(inputfile) == 1 && *inputfile == '-') {
- stream = (gpointer) stdin;
- } else if (inputfile) {
- if (!(stream = (gpointer) fopen (inputfile, "r"))) {
- fprintf(stderr, "unable to open log file\n");
- exit (-1);
- }
- } else if (!(stream = (gpointer) fopen (logfiles[i], "r"))) continue;
- } else {
- if (!(stream = (gpointer) gzopen (logfiles[i], "r"))) continue;
- }
-
- while (1) {
-
- if (parser->limit && (parser->count >= parser->limit)) {
- printf ("STATUS: aborted by limit (too many hits)\n");
- exit (0);
- }
-
- if (i <= 1) {
- line = fgets (linebuf, LINEBUFSIZE, (FILE *)stream);
- } else {
- line = mygzgets ((gzFile)stream, linebuf, LINEBUFSIZE);
- }
-
- if (!line) break;
-
- int len = strlen (line);
- int pid = 0;
-
- cpos = line;
-
- next_ctime = parse_time (&cpos, len);
-
- if (!next_ctime) {
- continue;
- }
-
- if (next_ctime != ctime) {
- rel_line_nr = 0;
- } else {
- rel_line_nr++;
- }
-
- ctime = next_ctime;
-
- if (ctime < start) continue;
- if (ctime > end) break;
-
- lines++;
-
- found = 0; while (!isspace (*cpos)) { cpos++; found = 1; }
- if (!found) {
- debug_error ("missing hostname", line);
- continue;
- }
-
- found = 0; while (isspace (*cpos)) { cpos++; found = 1; }
- if (!found) {
- debug_error ("missing spaces after host", line);
- continue;
- }
-
- if ((*cpos == 'l') && !strncmp (cpos, "last message repeated", 21)) {
- continue;
- }
-
- //printf ("LINE: %s\n", line);
- //const char *prog = cpos;
-
- csum_prog = 0;
- found = 0; while (*cpos && (*cpos != ':') && (*cpos != '[')) {
- csum_prog = ((csum_prog << 8)|(csum_prog >> 24)) + *cpos;
- cpos++;
- found++;
- }
-
- //idx1 = g_strndup (prog, found);
- //printf ("TEST:%s:%08X\n", idx1, csum_prog);
- //free (idx1);
-
- if (*cpos == '[') {
- cpos++;
- found = 0; while (isdigit (*cpos)) {
- pid = pid*10 + *cpos - '0';
- cpos++;
- found++;
- }
- if (found < 1 || found > 15 || *cpos != ']') {
- debug_error ("unable to parse pid", line);
- continue;
- }
- cpos++;
- }
-
- if (*cpos++ != ':') {
- debug_error ("missing collon", line);
- continue;
- }
-
-
- if (!isspace (*cpos++)) {
- debug_error ("missing space after collon", line);
- continue;
- }
-
- text = cpos;
-
- parser->ctime = ctime;
-
- int strmatch = 0;
-
- if (parser->strmatch && STRMATCH(parser->strmatch, text)) {
- strmatch = 1;
- }
-
- if ((csum_prog == PROXPROX) ||
- (csum_prog == PMG_SMTP_FILTER)) {
-
- if ((idx1 = parse_qid (&cpos, qidbuf, ':', 25))) {
-
- FEntry *fe;
-
- if (!(fe = fentry_get (parser, idx1))) {
- continue;
- }
-
- loglist_add (&fe->ep, &fe->loglist, line, len, lines);
-
- if (strmatch) fe->strmatch = 1;
-
- //fixme: BCC, Notify?
- //fixme: parse virus info
- //fixme: parse spam score
-
- if ((*cpos == 'a') && !strncmp (cpos, "accept mail to <", 16)) {
-
- const char *to_s, *to_e;
-
- to_s = cpos = cpos + 16;
-
- while (*cpos && (*cpos != '>')) { cpos++; }
-
- if (*cpos != '>') continue;
-
- to_e = cpos;
-
- cpos++;
-
- if ((*cpos++ != ' ') || (*cpos++ != '(')) continue;
-
- if (!(idx1 = parse_qid (&cpos, qidbuf, ')', 15))) continue;
-
- // parser_track (parser, idx1, 1);
-
- fentry_tolist_add (fe, 'A', to_s, to_e - to_s, idx1, strlen (idx1));
-
- } else if ((*cpos == 'm') && !strncmp (cpos, "moved mail for <", 16)) {
- const char *to_s, *to_e;
-
- to_s = cpos = cpos + 16;
-
- while (*cpos && (*cpos != '>')) { cpos++; }
-
- to_e = cpos;
-
- if (strncmp (cpos, "> to ", 5)) continue;
- cpos += 5;
-
- if (!strncmp (cpos, "spam", 4)) {
- cpos += 4;
- } else if (!strncmp (cpos, "virus", 5)) {
- cpos += 5;
- } else {
- continue;
- }
-
- if (strncmp (cpos, " quarantine - ", 14)) continue;
- cpos += 14;
-
- if (!(idx1 = parse_qid (&cpos, qidbuf, 0, 25))) continue;
-
- fentry_tolist_add (fe, 'Q', to_s, to_e - to_s, idx1, strlen (idx1));
-
- } else if ((*cpos == 'b') && !strncmp (cpos, "block mail to <", 15)) {
-
- const char *to_s;
-
- to_s = cpos = cpos + 15;
-
- while (*cpos && (*cpos != '>')) { cpos++; }
-
- if (*cpos != '>') continue;
-
- fentry_tolist_add (fe, 'B', to_s, cpos - to_s, NULL, 0);
-
- } else if ((*cpos == 'p') && !strncmp (cpos, "processing time: ", 17)) {
- cpos += 17;
-
- sscanf (cpos, "%f", &fe->ptime);
-
- fe->finished = 1;
- }
-
- }
-
- } else if (csum_prog == POSTFIX_POSTSCREEN) {
-
- SEntry *se;
-
- if (!pid) {
- debug_error ("no pid for postscreen", line);
- continue;
- }
-
-
- if ((*text == 'N') && !strncmp (text, "NOQUEUE: reject: RCPT from ", 27)) {
-
- cpos = text + 27;
-
- if (!(idx1 = strstr (cpos, "; client ["))) continue;
-
- const char *client = cpos = idx1 + 10;
-
- while (*cpos && (*cpos != ']')) { cpos++; }
-
- const char *client_end = cpos;
-
- if (!(idx1 = strstr (cpos, "; from=<"))) continue;
-
- const char *from = cpos = idx1 + 8;
-
- while (*cpos && (*cpos != '>')) { cpos++; }
- idx1 = cpos;
-
- if ((*cpos == '>') && strncmp (cpos, ">, to=<", 7)) continue;
-
- const char *to = cpos = cpos + 7;
-
- while (*cpos && (*cpos != '>')) { cpos++; }
-
- if (*cpos != '>') continue;
-
- if (!(se = sentry_get (parser, pid, ctime, rel_line_nr))) {
- continue;
- }
-
- if (strmatch) se->strmatch = 1;
-
- loglist_add (&se->ep, &se->loglist, line, len, lines);
-
- sentry_nqlist_add (se, ctime, from, idx1 - from, to, cpos - to, 'N');
-
- sentry_set_connect (se, client, client_end - client);
-
- g_hash_table_remove (parser->smtpd_h, &se->pid);
-
- se->disconnect = 1;
-
- sentry_print (parser, se);
- sentry_free (parser, se);
- }
-
- } else if (csum_prog == POSTFIX_QMGR) {
-
- if ((idx2 = text) && (idx1 = parse_qid (&idx2, qidbuf, ':', 15))) {
-
- QEntry *qe;
-
- if (!(qe = qentry_get (parser, idx1))) {
- continue;
- }
-
- if (strmatch) qe->strmatch = 1;
-
- qe->cleanup = 1;
-
- loglist_add (&qe->ep, &qe->loglist, line, len, lines);
-
- if ((*idx2 == 'f') && !strncmp (idx2, "from=<", 6)) {
-
- cpos = idx2 = idx2 + 6;
- while (*cpos && (*cpos != '>')) { cpos++; }
-
- if (*cpos != '>') {
- debug_error ("unable to parse 'from' address", line);
- continue;
- }
-
- qentry_set_from (qe, idx2, cpos - idx2);
-
- cpos++;
-
- if ((*cpos == ',') && !strncmp (cpos, ", size=", 7)) {
- int size = 0;
- cpos += 7;
-
- while (isdigit (*cpos)) { size = size*10 + *cpos++ - '0'; }
-
- qe->size = size;
- }
-
- } else if ((*idx2 == 'r') && !strncmp (idx2, "removed\n", 8)) {
-
- qe->removed = 1;
-
- qentry_finalize (parser, qe);
- }
- }
-
- } else if ((csum_prog == POSTFIX_SMTP) ||
- (csum_prog == POSTFIX_LMTP) ||
- (csum_prog == POSTFIX_LOCAL) ||
- (csum_prog == POSTFIX_ERROR)) {
-
- int lmtp = (csum_prog == POSTFIX_LMTP);
-
- if ((cpos = text) && (idx1 = parse_qid (&cpos, qidbuf, ':', 15))) {
-
- QEntry *qe;
-
- if (!(qe = qentry_get (parser, idx1))) {
- continue;
- }
-
- if (strmatch) qe->strmatch = 1;
-
- qe->cleanup = 1;
-
- loglist_add (&qe->ep, &qe->loglist, line, len, lines);
-
- if (strncmp (cpos, "to=<", 4)) continue;
- cpos += 4;
-
- const char *to_s, *to_e;
-
- to_s = cpos;
-
- while (*cpos && (*cpos != '>')) { cpos++; }
-
- if (*cpos != '>') continue;
-
- to_e = cpos;
-
- cpos ++;
-
- if (!(cpos = strstr (cpos, ", relay="))) continue;
- cpos += 8;
-
- const char *relay_s, *relay_e;
-
- relay_s = cpos;
-
- while (*cpos && (*cpos != ',')) { cpos++; }
-
- if (*cpos != ',') continue;
-
- relay_e = cpos;
-
- if (!(idx1 = strstr (cpos + 1, ", dsn="))) continue;
-
- cpos = idx1 + 6;
-
- if (!isdigit (*cpos)) continue;
-
- char dstatus = *cpos;
-
- qentry_tolist_add (qe, ctime, dstatus, to_s, to_e - to_s,
- relay_s, relay_e - relay_s);
-
- if (!lmtp) continue; // filter always uses lmtp
-
- if (!(idx1 = strstr (cpos + 1, "status=sent (250 2.")))
- continue;
-
- cpos = idx1 = idx1 + 19;
-
- if (*cpos == '5' && (idx1 = strstr (cpos, "5.0 OK ("))) {
- cpos = idx1 = idx1 + 8;
- } else if (*cpos == '7' && (idx1 = strstr (cpos, "7.0 BLOCKED ("))) {
- cpos = idx1 = idx1 + 13;
- } else {
- continue;
- }
-
- if (!(idx1 = parse_qid (&cpos, qidbuf, ')', 25)))
- continue;
-
- FEntry *fe;
-
- qe->filtered = 1;
-
- if ((fe = g_hash_table_lookup (parser->filter_h, idx1))) {
- qe->filter = fe;
- }
- }
-
- } else if (csum_prog == POSTFIX_SMTPD) {
- SEntry *se;
-
- if (!pid) {
- debug_error ("no pid for smtpd", line);
- continue;
- }
-
- if (!(se = sentry_get (parser, pid, ctime, rel_line_nr))) {
- continue;
- }
-
- if (strmatch) se->strmatch = 1;
-
- loglist_add (&se->ep, &se->loglist, line, len, lines);
-
- if ((*text == 'c') && !strncmp (text, "connect from ", 13)) {
-
- cpos = idx1 = text + 13;
-
- while (*idx1 && !isspace (*idx1)) { idx1++; }
-
- sentry_set_connect (se, cpos, idx1 - cpos);
-
- // fixme: do we need this?
- //if (strcmp (se->connect, "localhost[127.0.0.1]")) {
- // se->external = 1;
- //}
-
- } else if ((*text == 'd') && !strncmp (text, "disconnect from", 15)) {
-
- // unlink
- g_hash_table_remove (parser->smtpd_h, &se->pid);
-
- se->disconnect = 1;
-
- if (sentry_ref_rem_unneeded (parser, se) == 0) {
- sentry_print (parser, se);
- sentry_free (parser, se);
- } else {
- sentry_ref_finalize (parser, se);
- }
-
- } else if ((*text == 'N') && !strncmp (text, "NOQUEUE:", 8)) {
-
- cpos = text + 8;
-
- // parse 'whatsup' (reject:)
- while (*cpos && (*cpos != ':')) { cpos++; }
- if (*cpos != ':') continue;
- cpos++;
-
- // parse '%s from %s:'
- while (*cpos && (*cpos != ':')) { cpos++; }
- if (*cpos != ':') continue;
- idx1 = cpos++;
-
- // parse '%s;'
- while (*cpos && (*cpos != ';')) { cpos++; }
- if (*cpos != ';') continue;
-
- // greylisting test
- char dstatus = 'N';
- *(char *)cpos = 0; // dangerous hack: delimit string
- if (strstr (idx1, ": Recipient address rejected: Service is unavailable (try later)")) {
- dstatus = 'G';
- }
- *(char *)cpos = ';'; // dangerous hack: restore line
-
- if (!(idx1 = strstr (cpos, "; from=<"))) continue;
-
- const char *from = cpos = idx1 + 8;
-
- while (*cpos && (*cpos != '>')) { cpos++; }
- idx1 = cpos;
-
- if ((*cpos == '>') && strncmp (cpos, "> to=<", 6)) continue;
-
- const char *to = cpos = cpos + 6;
-
- while (*cpos && (*cpos != '>')) { cpos++; }
-
- if (*cpos != '>') continue;
-
- sentry_nqlist_add (se, ctime, from, idx1 - from, to, cpos - to, dstatus);
-
- } else if ((idx2 = text) && (idx1 = parse_qid (&idx2, qidbuf, ':', 15))) {
-
- QEntry *qe;
-
- if ((qe = qentry_get (parser, idx1))) {
-
- if (strmatch) qe->strmatch = 1;
-
- sentry_ref_add (se, qe);
-
- if ((*idx2 == 'c') && !strncmp (idx2, "client=", 7)) {
- cpos = idx2 = idx2 + 7;
-
- while (*cpos && !isspace (*cpos)) { cpos++; }
-
- qentry_set_client (qe, idx2, cpos - idx2);
- }
- }
-
- }
-
- } else if (csum_prog == POSTFIX_CLEANUP) { // postfix/cleanup
-
- QEntry *qe;
-
- idx2 = text;
- if ((idx1 = parse_qid (&idx2, qidbuf, ':', 15))) {
- if ((qe = qentry_get (parser, idx1))) {
-
- if (strmatch) qe->strmatch = 1;
-
- loglist_add (&qe->ep, &qe->loglist, line, len, lines);
-
- if ((*idx2 == 'm') && !strncmp (idx2, "message-id=", 11)) {
-
- cpos = idx2 = idx2 + 11;
-
- while (*cpos && !isspace(*cpos)) { cpos++; }
-
- qentry_set_msgid (qe, idx2, cpos - idx2);
-
- qe->cleanup = 1;
- }
- }
- }
- }
- }
-
- if (i <= 1) {
- fclose ((FILE *)stream);
- } else {
- gzclose ((gzFile)stream);
- }
-
- if (ctime > end) break;
- }
-
-
-#ifdef DEBUG
- printf ("LINES: %d\n", lines);
- printf ("MEM SMTPD entries: %d\n", g_hash_table_size (parser->smtpd_h));
- printf ("MEM QMGR entries: %d\n", g_hash_table_size (parser->qmgr_h));
- printf ("MEM FILTER entries: %d\n", g_hash_table_size (parser->filter_h));
-
- printf ("MEMDEB SMTPD entries: %d %d\n",
- g_hash_table_size (smtpd_debug_alloc),
- g_hash_table_size (smtpd_debug_free));
- printf ("MEMDEB QMGR entries: %d %d\n",
- g_hash_table_size (qmgr_debug_alloc),
- g_hash_table_size (qmgr_debug_free));
- printf ("MEMDEB FILTER entries: %d %d\n",
- g_hash_table_size (filter_debug_alloc),
- g_hash_table_size (filter_debug_free));
-#endif
-
- g_hash_table_foreach (parser->qmgr_h, qentry_cleanup_hash, parser);
- g_hash_table_foreach (parser->smtpd_h, sentry_cleanup_hash, parser);
- g_hash_table_foreach (parser->filter_h, fentry_cleanup_hash, parser);
-
-#ifdef DEBUG
- printf ("MEMDEB SMTPD entries: %d %d\n",
- g_hash_table_size (smtpd_debug_alloc),
- g_hash_table_size (smtpd_debug_free));
- printf ("MEMDEB QMGR entries: %d %d\n",
- g_hash_table_size (qmgr_debug_alloc),
- g_hash_table_size (qmgr_debug_free));
- printf ("MEMDEB FILTER entries: %d %d\n",
- g_hash_table_size (filter_debug_alloc),
- g_hash_table_size (filter_debug_free));
-
- g_hash_table_foreach (smtpd_debug_alloc, sentry_debug_alloc, parser);
-
-#endif
-
-
-#ifdef EPOOL_DEBUG
- printf ("MEMMAX %d\n", ep_maxalloc);
-#endif
-
- parser_free (parser);
-
- fclose (stdout);
-
- exit (0);
-}
--
2.20.1
More information about the pmg-devel
mailing list