From fd5d315164fdb4ad8775186f98df10d5447fdec2 Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Sat, 2 Mar 2024 17:10:06 -0600 Subject: [PATCH 01/11] =?UTF-8?q?=F0=9F=93=9D=20Add=20README?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 148 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..0751900 --- /dev/null +++ b/README.md @@ -0,0 +1,148 @@ +# my-log + +`my-log` is a tool for generating and parsing log files for whatever you want. This is early in development. Check our Roadmap before for what's working. + +I originally wrote [DropLogger](https://github.com/goodevilgenius/droplogger) to serve this purpose. DropLogger was originally designed to work primarily with IFTTT and Dropbox. Due to IFTTT changing their service significantly since I originally signed up, I no longer use it. Even without IFTTT, DropLogger is a great tool. + +So, why did I decide to completely rewrite it? Mainly because DropLogger is written in Python. While Python is a great language, I haven't used it seriously for many years, and I didn't find a whole lot of motivation to add new features to DropLogger, due to this. But I've been working in go for the past six months, and have kind of fell in love with the language. I'd been considering a rewrite of DropLogger for a while, so I decided to help myself get more practice in go by rewriting DropLogger in it. + +So, how does this work? + +Currently, it mostly doesn't. `my-log` is still in its early stages, and DropLogger is still needed for most of the functionality. So, how will it work? + +## Log files + +We start with the individual log files. These were designed to be very flexible so that they could be written using a number of different tools. Originally, IFTTT recipes were created that would write to files in Dropbox, but this could be adapted to a number of other automation tools to automatically write as things happen. + +What things? Well, maybe you use Tasker to trigger an action when you get home. You might want to keep a log of whenever you arrive at your house. Or, maybe you use [Last.FM](https://www.last.fm/home) to keep track of your music listening habits, and you want to log whenever you listen to a music track. You could create a Zap in Zapier that responds to new scrobbles on Last.FM, and adds those scrobbles to a file. + +### Log format + +As I mentioned, the format is intended to be very easy to write. Here's a sample: + +``` +@begin January 12, 2024 at 2:34PM - Title +@key value +@longKey this entry is long, and +spans multiple lines +@number 4 +@bool true +@end +``` + +So, each entry starts with `@begin` and ends with `@end`. It must have a date and a title. It may also have additional data which is indicated by an `@` at the beginning of the line. If I were to convert this to JSON (which `my-log` can do for you), it would look like: + +```json +{ + "title": "Title", + "date": "2024-01-12T14:34:00Z", + "key": "value", + "longKey": "this entry is long, and\nspans multiple lines", + "number": 4, + "bool": true +} +``` + +A couple things to note: +- When outputting JSON, the date is converted to ISO-8601 format. The timezone used (if none was given in the original log) is your own local time. +- The newline in the `longKey` was preserved +- Different types are recognized and parsed correctly. It supports the following types: + + string (default) + + numbers + + boolean values (true or false) + + dates and times + + A null value (the string "null", "nil", "none", or "~") + + A raw JSON object/array + +Since the extra fields are optional, the simplest log entry can be on a single line. For example, you might have a log file called `notes.txt` with this: + +``` +@begin February 3, 2015 at 01:33PM - Remember to call Mom @end +@begin February 4, 2015 at 07:45AM - Breakfast today was great! @end +``` + +As JSON, that would be: + +```json +[{ + "title":"Rember to call Mom", + "date":"2015-02-03T13:33:00Z" +},{ + "title":"Breakfase today was great!", + "date":"2015-02-04T07:45:00Z" +}] +``` + +### Adding log entries + +As was previously noted, the idea is that you can figure out the best way for you to add to the log file. But, `my-log` also comes with a command to add them from the command line. Run `my-log drop --help` for instructions on how to use it. But, here's a few examples: + +```shell +my-log drop notes "Hello" +# Adds "@begin - Hello @end" to notes.txt file + +my-log drop -d "yesterday" calls "Talked with Jeremy" -f phone_number=+1-555-867-5309 +# If today is January 2, 2024, adds the follow entry to calls.txt +# @begin January 1, 2024 at 12:00:00AM UTC - Talked with Jeremy +# @phone_number +1-555-867-5309 @end + +my-log drop -d "1999-12-31T23:59:59Z" events "The end of the world" -f notes="As we know it" -j '{"artist":"R.E.M","slaps":true}' +# Adds the following entry to events.txt +# @begin December 31, 1999 at 11:59:59PM UTC - The end of the world +# @notes As we know it +# @artist R.E.M +# @slaps true @end +``` + +## Output + +This is a work in progress. More info coming soon. Short version is, we want to be able to output to multiple formats in multiple places. + +Check [DropLogger's documentation](https://github.com/goodevilgenius/droplogger?tab=readme-ov-file#output) for info on how we want to make it work. + +## Configuration + +We use a TOML file for configuration. The default location on Linux is ~/.config/my-log/config.toml. You can find the exact location by doing `my-log -h` and looking at the help for the `--config` flag. Running `my-log config` will save the default config file to the default location. The file is intended to be edited by hand. There is no mechanism within the program to modify the file, aside from saving the default one. + +The default one has comments to help you out, but here's the options: + +### `[input]` + +- `path`: The path to where the logs are located. This is usually ~/my-log, but if you want to store it in Dropbox, you might want it to be ~/Dropbox/my-log +- `ext`: The file extension for log files. This is usually txt, which makes it easier to work with multiple tools, but you can change it to log, or my-log, if you want. If you set it to an empty string, no extension will be used, which also means that when parsing the log files, it will look at all files in the folder. +- `recurse`: Whether to look in sub-folders. + +### `[output.which-one]` + +Each separate output has its own set of configuration. So, replace `which-one` with the output name. + +- `enabled`: if set to false, will skip that output when running. +- `config`: This is an output-specific set of settings + +#### `[output.stdout.config]` + +*This section may change in the near future. We're considering supporting multiple formats.* + +- `json`: Outputs as JSON + +## Roadmap + +- [x] `drop` command. This is functional, and supports all the features of `drop-a-log` + + [ ] Don't add an extra blank line before new entries + + [ ] Add a new line at the end +- [ ] Output log entries + + [ ] A single date + + [ ] a specific period of time + + [ ] filter to specific logs + + [ ] stdout + - [ ] plain text + - [ ] JSON + - [ ] YAML + - [ ] Other formats? Submit an issue! + + [ ] file output + - [ ] Any format that stdout supports + - [ ] Multiple formats at once + - [ ] RSS + - [ ] ATOM + + [ ] sqlite database +- [ ] Maybe: plug-in system to add formats or output destinations From 286ac4557dbfa0abca5e9912080d73d37a4df051 Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Sun, 3 Mar 2024 13:56:48 -0600 Subject: [PATCH 02/11] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20Add=20tools.WriteVal?= =?UTF-8?q?ue?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- models/meta.go | 33 +++---------------------- models/meta_test.go | 2 +- tools/write_buffer.go | 44 ++++++++++++++++++++++++++++++++++ tools/write_buffer_test.go | 49 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 97 insertions(+), 31 deletions(-) create mode 100644 tools/write_buffer.go create mode 100644 tools/write_buffer_test.go diff --git a/models/meta.go b/models/meta.go index c737f95..dba99b2 100644 --- a/models/meta.go +++ b/models/meta.go @@ -2,12 +2,9 @@ package models import ( "bytes" - "encoding/json" "errors" "fmt" "regexp" - "strconv" - "time" "codeberg.org/danjones000/my-log/tools" ) @@ -25,33 +22,9 @@ func (m Meta) MarshalText() ([]byte, error) { buff.WriteRune('@') buff.WriteString(m.Key) buff.WriteRune(' ') - switch v := m.Value.(type) { - default: - return nil, fmt.Errorf("Unknown type %T", v) - case nil: - return []byte{}, nil - case string: - buff.WriteString(v) - case int: - buff.WriteString(strconv.Itoa(v)) - case int64: - buff.WriteString(strconv.FormatInt(v, 10)) - case float64: - buff.WriteString(strconv.FormatFloat(v, 'f', -1, 64)) - case json.Number: - buff.WriteString(v.String()) - case json.RawMessage: - buff.Write(v) - case []byte: - buff.Write(v) - case byte: - buff.WriteByte(v) - case rune: - buff.WriteString(string(v)) - case bool: - buff.WriteString(strconv.FormatBool(v)) - case time.Time: - buff.WriteString(v.Format(time.RFC3339)) + n, err := tools.WriteValue(buff, m.Value) + if n == 0 || err != nil { + return []byte{}, err } return buff.Bytes(), nil diff --git a/models/meta_test.go b/models/meta_test.go index 2dae977..1e0888b 100644 --- a/models/meta_test.go +++ b/models/meta_test.go @@ -40,7 +40,7 @@ func TestMeta(t *testing.T) { {"byte", "byteme", byte(67), "@byteme C", nil, "C"}, {"json-obj", "obj", json.RawMessage(`{"foo":"bar","baz":"quux"}`), `@obj {"foo":"bar","baz":"quux"}`, nil, json.RawMessage(`{"foo":"bar","baz":"quux"}`)}, {"json-arr", "arr", json.RawMessage(`["foo",42,"bar", null,"quux", true]`), `@arr ["foo",42,"bar", null,"quux", true]`, nil, json.RawMessage(`["foo",42,"bar", null,"quux", true]`)}, - {"chan", "nope", make(chan bool), "", errors.New("Unknown type chan bool"), ""}, + {"chan", "nope", make(chan bool), "", errors.New("Unsupported type chan bool"), ""}, {"whitespace-key", "no space", "hi", "", errors.New("whitespace is not allowed in key: no space"), ""}, {"empty-mar", "nope", skipMarshalTest, "", nil, ErrorParsing}, {"no-key-mar", "nope", skipMarshalTest, "nope", nil, ErrorParsing}, diff --git a/tools/write_buffer.go b/tools/write_buffer.go new file mode 100644 index 0000000..e67d219 --- /dev/null +++ b/tools/write_buffer.go @@ -0,0 +1,44 @@ +package tools + +import ( + "bytes" + "encoding/json" + "fmt" + "strconv" + "time" +) + +func WriteValue(buff *bytes.Buffer, val any) (n int, err error) { + switch v := val.(type) { + default: + err = fmt.Errorf("Unsupported type %T", v) + case nil: + return + case string: + return buff.WriteString(v) + case int: + return buff.WriteString(strconv.Itoa(v)) + case int64: + return buff.WriteString(strconv.FormatInt(v, 10)) + case float64: + return buff.WriteString(strconv.FormatFloat(v, 'f', -1, 64)) + case json.Number: + return buff.WriteString(v.String()) + case json.RawMessage: + return buff.Write(v) + case []byte: + return buff.Write(v) + case byte: + err = buff.WriteByte(v) + if err == nil { + n = 1 + } + case rune: + return buff.WriteString(string(v)) + case bool: + return buff.WriteString(strconv.FormatBool(v)) + case time.Time: + return buff.WriteString(v.Format(time.RFC3339)) + } + return +} diff --git a/tools/write_buffer_test.go b/tools/write_buffer_test.go new file mode 100644 index 0000000..1a4a103 --- /dev/null +++ b/tools/write_buffer_test.go @@ -0,0 +1,49 @@ +package tools + +import ( + "bytes" + "encoding/json" + "errors" + "testing" + "time" + + "github.com/stretchr/testify/assert" +) + +func TestWriteBuffer(t *testing.T) { + when := time.Now() + tests := []struct { + name string + value any + out string + err error + }{ + {"nil", nil, "", nil}, + {"string", "hi", "hi", nil}, + {"bytes", []byte{104, 105}, "hi", nil}, + {"byte", byte(104), "h", nil}, + {"rune", 'h', "h", nil}, + {"int", 42, "42", nil}, + {"int64", int64(42), "42", nil}, + {"float", 42.13, "42.13", nil}, + {"bool", false, "false", nil}, + {"json.Number", json.Number("42.13"), "42.13", nil}, + {"json.RawMessage", json.RawMessage("{}"), "{}", nil}, + {"time", when, when.Format(time.RFC3339), nil}, + {"struct", struct{}{}, "", errors.New("Unsupported type struct {}")}, + } + + for _, tt := range tests { + t.Run(tt.name, getWriteTestRunner(tt.value, tt.out, tt.err)) + } +} + +func getWriteTestRunner(value any, out string, err error) func(*testing.T) { + return func(t *testing.T) { + buff := &bytes.Buffer{} + n, er := WriteValue(buff, value) + assert.Equal(t, len(out), n) + assert.Equal(t, err, er) + assert.Equal(t, out, buff.String()) + } +} From 89e6c2b3bd961dd2af40b45aee607f53237cdcc2 Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Thu, 7 Mar 2024 10:10:54 -0600 Subject: [PATCH 03/11] =?UTF-8?q?=E2=9C=A8=20Add=20plain=20text=20formatte?= =?UTF-8?q?r?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- formatters/interface.go | 10 ++++ formatters/new.go | 34 ++++++++++++ formatters/new_test.go | 35 +++++++++++++ formatters/plain.go | 88 +++++++++++++++++++++++++++++++ formatters/plain_test.go | 109 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 276 insertions(+) create mode 100644 formatters/interface.go create mode 100644 formatters/new.go create mode 100644 formatters/new_test.go create mode 100644 formatters/plain.go create mode 100644 formatters/plain_test.go diff --git a/formatters/interface.go b/formatters/interface.go new file mode 100644 index 0000000..4276aed --- /dev/null +++ b/formatters/interface.go @@ -0,0 +1,10 @@ +package formatters + +import "codeberg.org/danjones000/my-log/models" + +type Formatter interface { + Name() string + Log(models.Log) (out []byte, err error) + Entry(models.Entry) (out []byte, err error) + Meta(models.Meta) (out []byte, err error) +} diff --git a/formatters/new.go b/formatters/new.go new file mode 100644 index 0000000..8cd567f --- /dev/null +++ b/formatters/new.go @@ -0,0 +1,34 @@ +package formatters + +import ( + "errors" + + "codeberg.org/danjones000/my-log/config" +) + +type formatMaker func(oo config.Outputs) (Formatter, error) + +var formatterMap = map[string]formatMaker{ + "plain": newPlain, +} + +func New(kind string) (f Formatter, err error) { + conf, err := config.Load() + if err != nil { + return + } + + if make, ok := formatterMap[kind]; ok { + return make(conf.Outputs) + } + + return nil, errors.New("unimplemented") +} + +func Kinds() []string { + r := []string{} + for kind, _ := range formatterMap { + r = append(r, kind) + } + return r +} diff --git a/formatters/new_test.go b/formatters/new_test.go new file mode 100644 index 0000000..bf91000 --- /dev/null +++ b/formatters/new_test.go @@ -0,0 +1,35 @@ +package formatters + +import ( + "fmt" + "os" + "testing" + + "codeberg.org/danjones000/my-log/config" + "github.com/stretchr/testify/assert" +) + +func TestKinds(t *testing.T) { + assert.Equal(t, []string{"plain"}, Kinds()) +} + +func TestNewUnsupported(t *testing.T) { + f, err := New("nope") + assert.Nil(t, f) + assert.Error(t, err) +} + +func TestNewCantGetConfig(t *testing.T) { + f, _ := os.CreateTemp("", "test") + oldConf := config.ConfigPath + config.ConfigPath = f.Name() + defer f.Close() + defer func() { + config.ConfigPath = oldConf + }() + + fmt.Fprint(f, `{"not":"toml"}`) + form, err := New("plain") + assert.Nil(t, form) + assert.Error(t, err) +} diff --git a/formatters/plain.go b/formatters/plain.go new file mode 100644 index 0000000..420b008 --- /dev/null +++ b/formatters/plain.go @@ -0,0 +1,88 @@ +package formatters + +import ( + "bytes" + + "codeberg.org/danjones000/my-log/config" + "codeberg.org/danjones000/my-log/models" + "codeberg.org/danjones000/my-log/tools" +) + +func newPlain(oo config.Outputs) (Formatter, error) { + return &PlainText{}, nil +} + +type PlainText struct { + // config might go here some day +} + +func (pt *PlainText) Name() string { + return "plain" +} + +func (pt *PlainText) Log(log models.Log) (out []byte, err error) { + if len(log.Entries) == 0 { + return + } + + buff := &bytes.Buffer{} + buff.WriteString(log.Name) + buff.WriteString("\n#######") + written := false + for _, e := range log.Entries { + bb := pt.entryBuffer(e) + if bb.Len() > 0 { + buff.WriteByte(10) + buff.WriteByte(10) + buff.ReadFrom(bb) + written = true + } + } + if written { + out = buff.Bytes() + } + return +} + +func (pt *PlainText) entryBuffer(entry models.Entry) *bytes.Buffer { + buff := &bytes.Buffer{} + buff.WriteString("Title: ") + buff.WriteString(entry.Title) + buff.WriteByte(10) + buff.WriteString("Date: ") + buff.WriteString(entry.Date.Format(tools.DateFormat)) + for _, m := range entry.Fields { + bb, err := pt.metaBuffer(m) + if (bb.Len() > 0) && (err == nil) { + buff.WriteByte(10) + buff.ReadFrom(bb) + } + } + + return buff +} + +func (pt *PlainText) Entry(entry models.Entry) ([]byte, error) { + buff := pt.entryBuffer(entry) + return buff.Bytes(), nil +} + +func (pt *PlainText) metaBuffer(meta models.Meta) (*bytes.Buffer, error) { + buff := &bytes.Buffer{} + buff.WriteString(meta.Key) + buff.WriteString(": ") + n, err := tools.WriteValue(buff, meta.Value) + if n == 0 || err != nil { + return &bytes.Buffer{}, err + } + return buff, nil +} + +func (pt *PlainText) Meta(meta models.Meta) (out []byte, err error) { + buff, err := pt.metaBuffer(meta) + if err != nil { + return + } + out = buff.Bytes() + return +} diff --git a/formatters/plain_test.go b/formatters/plain_test.go new file mode 100644 index 0000000..e384101 --- /dev/null +++ b/formatters/plain_test.go @@ -0,0 +1,109 @@ +package formatters + +import ( + "bufio" + "bytes" + "fmt" + "testing" + "time" + + "codeberg.org/danjones000/my-log/models" + "codeberg.org/danjones000/my-log/tools" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestPlainLog(t *testing.T) { + m := []models.Meta{ + {"foo", "bar"}, + {"baz", 42}, + } + e := []models.Entry{ + {Title: "one", Date: time.Now(), Fields: m}, + {Title: "small", Date: time.Now()}, + } + l := models.Log{"stuff", e} + + f, err := New("plain") + require.NoError(t, err) + + out, err := f.Log(l) + require.NoError(t, err) + + read := bytes.NewReader(out) + scan := bufio.NewScanner(read) + + scan.Scan() + line := scan.Text() + assert.Equal(t, l.Name, line) + + scan.Scan() + line = scan.Text() + assert.Equal(t, "#######", line) + + scan.Scan() + scan.Scan() + line = scan.Text() + assert.Equal(t, "Title: "+e[0].Title, line) + + scan.Scan() + line = scan.Text() + assert.Equal(t, "Date: "+e[0].Date.Format(tools.DateFormat), line) + + scan.Scan() + line = scan.Text() + assert.Equal(t, "foo: bar", line) + + scan.Scan() + line = scan.Text() + assert.Equal(t, "baz: 42", line) + + scan.Scan() + scan.Scan() + line = scan.Text() + assert.Equal(t, "Title: "+e[1].Title, line) + + scan.Scan() + line = scan.Text() + assert.Equal(t, "Date: "+e[1].Date.Format(tools.DateFormat), line) + + more := scan.Scan() + assert.False(t, more) +} + +func TestPlainName(t *testing.T) { + f, _ := New("plain") + assert.Equal(t, "plain", f.Name()) +} + +func TestPlainLogNoEntries(t *testing.T) { + f, _ := New("plain") + out, err := f.Log(models.Log{Name: "foo"}) + assert.NoError(t, err) + assert.Len(t, out, 0) +} + +func TestPlainMetaEmpty(t *testing.T) { + f, _ := New("plain") + out, err := f.Meta(models.Meta{"foo", ""}) + assert.NoError(t, err) + assert.Len(t, out, 0) +} + +func TestPlainMetaError(t *testing.T) { + f, _ := New("plain") + out, err := f.Meta(models.Meta{"foo", make(chan bool)}) + assert.Error(t, err) + assert.Len(t, out, 0) +} + +func TestPlainEntry(t *testing.T) { + f, _ := New("plain") + now := time.Now() + out, err := f.Entry(models.Entry{ + Title: "foo", + Date: now, + }) + assert.NoError(t, err) + assert.Equal(t, fmt.Sprintf("Title: foo\nDate: %s", now.Format(tools.DateFormat)), string(out)) +} From 99f6dc3f8cabcda89adf31a21d33ad4af7759984 Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Thu, 7 Mar 2024 10:48:41 -0600 Subject: [PATCH 04/11] =?UTF-8?q?=E2=9C=A8=20Use=20plain=20formatter=20to?= =?UTF-8?q?=20output=20entry=20from=20drop?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- cmd/drop.go | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/cmd/drop.go b/cmd/drop.go index f7fc43d..2b33750 100644 --- a/cmd/drop.go +++ b/cmd/drop.go @@ -22,6 +22,7 @@ import ( "time" "codeberg.org/danjones000/my-log/files" + "codeberg.org/danjones000/my-log/formatters" "codeberg.org/danjones000/my-log/models" "codeberg.org/danjones000/my-log/tools" "github.com/spf13/cobra" @@ -58,11 +59,17 @@ var dropCmd = &cobra.Command{ if err != nil { return err } - by, err := e.MarshalText() + + form, err := formatters.New("plain") if err != nil { return err } - fmt.Fprintf(cmd.OutOrStdout(), "%s\n", by) + out, err := form.Log(l) + if err != nil { + return err + } + fmt.Fprintf(cmd.OutOrStdout(), "%s\n", out) + return nil }, } From da3b5249259b4b1b879eb9aebb120de83fb8418a Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Thu, 7 Mar 2024 21:19:45 -0600 Subject: [PATCH 05/11] =?UTF-8?q?=E2=9C=A8=20Separate=20formatters=20in=20?= =?UTF-8?q?config?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- config/default.go | 10 ++++++++-- config/load.go | 19 +++++++++++++++++++ config/load_test.go | 19 ++++++++++++++++--- config/types.go | 13 ++++++++++--- formatters/new.go | 4 ++-- formatters/plain.go | 2 +- 6 files changed, 56 insertions(+), 11 deletions(-) diff --git a/config/default.go b/config/default.go index 461f5c3..76ef8d2 100644 --- a/config/default.go +++ b/config/default.go @@ -27,8 +27,14 @@ dotFolder = true [output.stdout] enabled = true [output.stdout.config] -# Whether to output as JSON. Maybe useful to pipe elsewhere. -json = false +# Formatter to use when outputting to stdout +formatter = "plain" + +[formatters] + +[formatters.json] +# Set to true to pretty print JSON output +pretty_print = false ` diff --git a/config/load.go b/config/load.go index 04fd95d..3de0a19 100644 --- a/config/load.go +++ b/config/load.go @@ -32,6 +32,7 @@ func Load() (Config, error) { } env.Parse(&c) c.Outputs["stdout"] = loadStdout(c.Outputs["stdout"]) + c.Formatters["json"] = loadJsonFormat(c.Formatters["json"]) l := "" for k, v := range Overrides { @@ -77,3 +78,21 @@ func (oo Outputs) Stdout() (s Stdout, enabled bool) { return } + +func loadJsonFormat(c map[string]any) map[string]any { + jf := JsonFormat{} + mapst.Decode(c, &jf) + env.Parse(&jf) + mapst.Decode(jf, &c) + return c +} + +func (ff Formatters) Json() (jf JsonFormat) { + o, ok := ff["json"] + if !ok { + return + } + + mapst.Decode(o, &jf) + return +} diff --git a/config/load_test.go b/config/load_test.go index 7399316..ca6a0b7 100644 --- a/config/load_test.go +++ b/config/load_test.go @@ -65,12 +65,25 @@ func TestStdoutMissing(t *testing.T) { } func TestStdoutLoad(t *testing.T) { - os.Setenv("LOG_STDOUT_JSON", "true") - defer os.Unsetenv("LOG_STDOUT_JSON") + os.Setenv("LOG_STDOUT_FORMATTER", "json") + defer os.Unsetenv("LOG_STDOUT_FORMATTER") os.Setenv("LOG_STDOUT_ENABLED", "true") defer os.Unsetenv("LOG_STDOUT_ENABLED") c, _ := Load() std, en := c.Outputs.Stdout() assert.True(t, en) - assert.True(t, std.Json) + assert.Equal(t, "json", std.Formatter) +} + +func TestFormatJson(t *testing.T) { + ff := Formatters{ + "json": map[string]any{"pretty_print": true}, + } + + js := ff.Json() + assert.True(t, js.PrettyPrint) + + ff = Formatters{} + js = ff.Json() + assert.False(t, js.PrettyPrint) } diff --git a/config/types.go b/config/types.go index a3a3681..a950a80 100644 --- a/config/types.go +++ b/config/types.go @@ -1,8 +1,9 @@ package config type Config struct { - Input Input - Outputs Outputs `toml:"output"` + Input Input + Outputs Outputs `toml:"output"` + Formatters Formatters } type Input struct { @@ -20,9 +21,15 @@ type Output struct { } type Stdout struct { - Json bool `env:"LOG_STDOUT_JSON" mapstructure:"json"` + Formatter string `env:"LOG_STDOUT_FORMATTER" mapstructure:"formatter"` } type stdoutEnabled struct { Enabled bool `env:"LOG_STDOUT_ENABLED"` } + +type Formatters map[string]map[string]any + +type JsonFormat struct { + PrettyPrint bool `env:"LOG_JSON_PRETTY_PRINT" mapstructure:"pretty_print"` +} diff --git a/formatters/new.go b/formatters/new.go index 8cd567f..33084e3 100644 --- a/formatters/new.go +++ b/formatters/new.go @@ -6,7 +6,7 @@ import ( "codeberg.org/danjones000/my-log/config" ) -type formatMaker func(oo config.Outputs) (Formatter, error) +type formatMaker func(config.Formatters) (Formatter, error) var formatterMap = map[string]formatMaker{ "plain": newPlain, @@ -19,7 +19,7 @@ func New(kind string) (f Formatter, err error) { } if make, ok := formatterMap[kind]; ok { - return make(conf.Outputs) + return make(conf.Formatters) } return nil, errors.New("unimplemented") diff --git a/formatters/plain.go b/formatters/plain.go index 420b008..ed8ee82 100644 --- a/formatters/plain.go +++ b/formatters/plain.go @@ -8,7 +8,7 @@ import ( "codeberg.org/danjones000/my-log/tools" ) -func newPlain(oo config.Outputs) (Formatter, error) { +func newPlain(ff config.Formatters) (Formatter, error) { return &PlainText{}, nil } From f68aebdedb9033b9106d0b1dc2c4e84222d58c6e Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Thu, 7 Mar 2024 21:50:51 -0600 Subject: [PATCH 06/11] =?UTF-8?q?=E2=9C=85=20=F0=9F=92=AF%?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- config/load_test.go | 7 ++++++- files/append_test.go | 15 +++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/config/load_test.go b/config/load_test.go index ca6a0b7..51e024e 100644 --- a/config/load_test.go +++ b/config/load_test.go @@ -55,7 +55,12 @@ func TestOverrideJson(t *testing.T) { assert.Equal(t, "txt", c.Input.Ext) } -// @todo test time +func TestTimeParse(t *testing.T) { + Overrides = map[string]string{"input.ext": "now"} + c, err := Load() + assert.ErrorContains(t, err, "incompatible types: TOML value has type time.Time; destination has type string") + assert.Equal(t, "txt", c.Input.Ext) +} func TestStdoutMissing(t *testing.T) { var oo Outputs = map[string]Output{} diff --git a/files/append_test.go b/files/append_test.go index 487acfd..c31ec63 100644 --- a/files/append_test.go +++ b/files/append_test.go @@ -57,6 +57,21 @@ func (s *AppendTestSuite) TestSuccess() { s.Assert().Contains(st, "\n@bar true") } +func (s *AppendTestSuite) TestFailEntry() { + e := models.Entry{ + Title: "Jimmy", + } + l := models.Log{ + Name: "test", + Entries: []models.Entry{e}, + } + err := Append(l) + s.Assert().NoError(err) + s.Require().FileExists(s.dir + "/test.log") + by, _ := os.ReadFile(s.dir + "/test.log") + s.Assert().Equal([]byte{}, by) +} + func (s *AppendTestSuite) TestDotFolder() { config.Overrides["input.dotFolder"] = "true" e := models.Entry{ From 5b8e4696ea7381ef24b2ca7d382a34bc087fd769 Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Sat, 9 Mar 2024 15:38:34 -0600 Subject: [PATCH 07/11] =?UTF-8?q?=E2=9C=A8=20Only=20add=20newline=20to=20f?= =?UTF-8?q?ile=20when=20needed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- files/append.go | 17 +++++++++++- files/append_test.go | 62 ++++++++++++++++++++++++++++++++++++++++++++ models/entry.go | 2 +- models/entry_test.go | 6 ++--- 4 files changed, 82 insertions(+), 5 deletions(-) diff --git a/files/append.go b/files/append.go index bd9f4ee..cb62d9b 100644 --- a/files/append.go +++ b/files/append.go @@ -2,6 +2,7 @@ package files import ( "fmt" + "io" "os" fp "path/filepath" "strings" @@ -31,18 +32,32 @@ func Append(l models.Log) error { return err } - f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0640) + f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_RDWR, 0640) if err != nil { return err } defer f.Close() + f.Seek(-1, os.SEEK_END) + last := make([]byte, 1, 1) + n, err := f.Read(last) + if err != nil && err != io.EOF { + return err + } + + if err == nil && n > 0 { + if last[0] != 10 { + f.Write([]byte{10}) + } + } + for _, e := range l.Entries { by, err := e.MarshalText() if err != nil { continue } f.Write(by) + f.Write([]byte{10}) } return nil diff --git a/files/append_test.go b/files/append_test.go index c31ec63..0976363 100644 --- a/files/append_test.go +++ b/files/append_test.go @@ -33,6 +33,7 @@ func (s *AppendTestSuite) TearDownSuite() { } func (s *AppendTestSuite) TestSuccess() { + defer os.Remove(s.dir + "/test.log") when := time.Now().Local() e := models.Entry{ Title: "Jimmy", @@ -57,7 +58,68 @@ func (s *AppendTestSuite) TestSuccess() { s.Assert().Contains(st, "\n@bar true") } +func (s *AppendTestSuite) TestTwoEntries() { + defer os.Remove(s.dir + "/test.log") + when := time.Now().Local() + whens := when.Format(models.DateFormat) + e := []models.Entry{ + {Title: "one", Date: when}, + {Title: "two", Date: when}, + } + l := models.Log{ + Name: "test", + Entries: e, + } + err := Append(l) + s.Assert().NoError(err) + s.Require().FileExists(s.dir + "/test.log") + by, _ := os.ReadFile(s.dir + "/test.log") + exp := fmt.Sprintf("@begin %s - one @end\n@begin %s - two @end\n", whens, whens) + s.Assert().Equal(exp, string(by)) +} + +func (s *AppendTestSuite) TestAddNewLine() { + defer os.Remove(s.dir + "/test.log") + os.WriteFile(s.dir+"/test.log", []byte("foo"), 0644) + when := time.Now().Local() + whens := when.Format(models.DateFormat) + e := []models.Entry{ + {Title: "one", Date: when}, + } + l := models.Log{ + Name: "test", + Entries: e, + } + err := Append(l) + s.Assert().NoError(err) + s.Require().FileExists(s.dir + "/test.log") + by, _ := os.ReadFile(s.dir + "/test.log") + exp := fmt.Sprintf("foo\n@begin %s - one @end\n", whens) + s.Assert().Equal(exp, string(by)) +} + +func (s *AppendTestSuite) TestDontAddNewLine() { + defer os.Remove(s.dir + "/test.log") + os.WriteFile(s.dir+"/test.log", []byte("foo\n"), 0644) + when := time.Now().Local() + whens := when.Format(models.DateFormat) + e := []models.Entry{ + {Title: "one", Date: when}, + } + l := models.Log{ + Name: "test", + Entries: e, + } + err := Append(l) + s.Assert().NoError(err) + s.Require().FileExists(s.dir + "/test.log") + by, _ := os.ReadFile(s.dir + "/test.log") + exp := fmt.Sprintf("foo\n@begin %s - one @end\n", whens) + s.Assert().Equal(exp, string(by)) +} + func (s *AppendTestSuite) TestFailEntry() { + defer os.Remove(s.dir + "/test.log") e := models.Entry{ Title: "Jimmy", } diff --git a/models/entry.go b/models/entry.go index d504536..5a19933 100644 --- a/models/entry.go +++ b/models/entry.go @@ -74,7 +74,7 @@ func (e Entry) MarshalText() ([]byte, error) { } ch := e.getFieldMarshalChan() buff := &bytes.Buffer{} - buff.WriteString("\n@begin ") + buff.WriteString("@begin ") buff.WriteString(e.Date.Format(DateFormat)) buff.WriteString(" - ") buff.WriteString(e.Title) diff --git a/models/entry_test.go b/models/entry_test.go index a8bec44..aefecc9 100644 --- a/models/entry_test.go +++ b/models/entry_test.go @@ -90,13 +90,13 @@ func getEntryMarshalTestRunner(title string, date time.Time, fields []Meta, firs if first == "" { return } + os := string(o) if len(lines) == 0 { - assert.Equal(t, "\n"+first, string(o)) + assert.Equal(t, first, os) return } - os := string(o) - assert.Regexp(t, "^\n"+first, os) + assert.Regexp(t, first, os) for _, line := range lines { assert.Regexp(t, "(?m)^"+line, os) } From 11dea95ce2b27cb45ac1486e5c7f76249e48c237 Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Sat, 9 Mar 2024 15:42:29 -0600 Subject: [PATCH 08/11] =?UTF-8?q?=F0=9F=93=9D=20Update=20README=20with=20n?= =?UTF-8?q?ew=20features?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 0751900..15d8a20 100644 --- a/README.md +++ b/README.md @@ -128,14 +128,14 @@ Each separate output has its own set of configuration. So, replace `which-one` w ## Roadmap - [x] `drop` command. This is functional, and supports all the features of `drop-a-log` - + [ ] Don't add an extra blank line before new entries - + [ ] Add a new line at the end + + [x] Don't add an extra blank line before new entries + + [x] Add a new line at the end - [ ] Output log entries + [ ] A single date + [ ] a specific period of time + [ ] filter to specific logs + [ ] stdout - - [ ] plain text + - [x] plain text - [ ] JSON - [ ] YAML - [ ] Other formats? Submit an issue! From febbce8a6be26e7057fa2368c492ea7acf54cbb4 Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Sat, 9 Mar 2024 16:05:59 -0600 Subject: [PATCH 09/11] =?UTF-8?q?=E2=9C=A8=20Add=20formatters.Preferred?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- cmd/drop.go | 2 +- cmd/root.go | 2 +- formatters/new.go | 10 ++++++++++ formatters/new_test.go | 10 ++++++++++ 4 files changed, 22 insertions(+), 2 deletions(-) diff --git a/cmd/drop.go b/cmd/drop.go index 2b33750..658cb7c 100644 --- a/cmd/drop.go +++ b/cmd/drop.go @@ -60,7 +60,7 @@ var dropCmd = &cobra.Command{ return err } - form, err := formatters.New("plain") + form, err := formatters.Preferred() if err != nil { return err } diff --git a/cmd/root.go b/cmd/root.go index 954a3a7..7e7bbbd 100644 --- a/cmd/root.go +++ b/cmd/root.go @@ -50,7 +50,7 @@ func init() { // will be global for your application. rootCmd.PersistentFlags().StringVarP(&config.ConfigPath, "config", "c", config.ConfigPath, "config file") - rootCmd.PersistentFlags().StringToStringVarP(&config.Overrides, "config-value", "v", config.Overrides, "Override config values. Use dot syntax to specify key. E.g. -v output.stdout.config.json=true") + rootCmd.PersistentFlags().StringToStringVarP(&config.Overrides, "config-value", "v", config.Overrides, "Override config values. Use dot syntax to specify key. E.g. -v output.stdout.config.formatter=json") // Cobra also supports local flags, which will only run // when this action is called directly. diff --git a/formatters/new.go b/formatters/new.go index 33084e3..d88a9d0 100644 --- a/formatters/new.go +++ b/formatters/new.go @@ -12,6 +12,16 @@ var formatterMap = map[string]formatMaker{ "plain": newPlain, } +func Preferred() (f Formatter, err error) { + conf, err := config.Load() + if err != nil { + return + } + std, _ := conf.Outputs.Stdout() + kind := std.Formatter + return New(kind) +} + func New(kind string) (f Formatter, err error) { conf, err := config.Load() if err != nil { diff --git a/formatters/new_test.go b/formatters/new_test.go index bf91000..04b40fb 100644 --- a/formatters/new_test.go +++ b/formatters/new_test.go @@ -32,4 +32,14 @@ func TestNewCantGetConfig(t *testing.T) { form, err := New("plain") assert.Nil(t, form) assert.Error(t, err) + + form, err = Preferred() + assert.Nil(t, form) + assert.Error(t, err) +} + +func TestPreferred(t *testing.T) { + form, err := Preferred() + assert.NotNil(t, form) + assert.NoError(t, err) } From 17da5b66ea4586440f302e8a29b52fb417304876 Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Sat, 9 Mar 2024 16:27:32 -0600 Subject: [PATCH 10/11] =?UTF-8?q?=F0=9F=93=9D=20Add=20formatter=20info=20t?= =?UTF-8?q?o=20README?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 15d8a20..24eae91 100644 --- a/README.md +++ b/README.md @@ -123,7 +123,15 @@ Each separate output has its own set of configuration. So, replace `which-one` w *This section may change in the near future. We're considering supporting multiple formats.* -- `json`: Outputs as JSON +- `formatter`: Which formatter to use when outputting data. This value is used by `my-log drop` to output the new entry. + +### `[formatters]` + +Some formatters may have custom configuration. + +#### `[formatters.json]` + +- `pretty_print`: If true, JSON output will be pretty printed. If false, it will be printed to a single line. ## Roadmap From 33fbdf7ecb06054d0a71bd21e8abb1387bd50339 Mon Sep 17 00:00:00 2001 From: Dan Jones Date: Sat, 9 Mar 2024 17:38:35 -0600 Subject: [PATCH 11/11] =?UTF-8?q?=F0=9F=93=9D=20Add=20to=20CHANGELOG?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CHANGELOG.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 122a538..30e3b58 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,10 @@ # Changelog +## [0.0.2] - 2024-03-09 + +- ✨ Use plain formatter to output entry from drop +- ✨ Add newline to file when needed + ## [0.0.1] - 2024-03-02 🎉 Initial release.