trying to get kal useful for me again

3 files changed, 222 insertions, 53 deletions

diff --git a/TASK_MANAGEMENT.md b/TASK_MANAGEMENT.md
index 1876aa9..cb2ae1b 100644
--- a/TASK_MANAGEMENT.md
+++ b/TASK_MANAGEMENT.md
@@ -90,41 +90,41 @@ I tried implementing some logic like this in calendar-cli, and it was working on
 
 There is no support for rrules outside the task completion code, so as for now the rrule has to be put in through another caldav client tool, through the --pdb option or through manually editing ical code.  I believe recurring tasks is an important functionality, so I will implement better support for this at some point.
 
-dtstart vs due vs duration vs priority
---------------------------------------
+dtstart vs due vs duration
+--------------------------
 
-Everything below describes my workflow as of 2015.  I have reconsidered and I'm working on new workflow - see the document [NEXT_LEVEL.md](NEXT_LEVEL.md).  The short summary: DURATION should be the actual time estimate, DUE is when you'd like to be done with the task, DTSTART is the latest possible time you can start with the task if the DUE is to be held, PRIORITY should show how urgent it is to complete before DUE (or complete the task at all), and it will be needed with slightly new logic for sorting and listing tasks.  I'm also planning to follow up with linked VJOURNAL-entries for keeping tabs on (potentially billable) time spent on work, as well as the possibility to link VEVENT and VTODO (to allocate specific time for working with a task, or mark up that some TODO needs to be done before some event)
+I don't know what they were thinking of when they created the icalendar standard.
 
-As of 2015 my opinion was that dtstart is the earliest time you expect to start working with the vtodo, or maybe the earliest time it's possible to start.  Passing the dtstart doesn't mean you need to drop everything else and start working on the task immediately.  You'd want to postpone the dtstart just to unclutter the todo-list.
+An event may have a DTSTART and a DUE ... or alternatively, a DURATION instead of DUE.  I assume the intention is that a task with DTSTART and DURATION set is equivalent with a task with the smae DTSTART set, and a DUE set equal to DTSTART plus DURATION.  This makes a lot of sense for events, but for tasks?  Not so much!
 
-Due is the time/date when the task has to be completed, come hell or high water.  It should probably not be postponed.  Due dates should probably be set very far in the future if no specific due date is set.  You really don't want the list of tasks that are soon due or even overdue to be cluttered up with stuff that can be procrastinated even further.
+Ok, DUE is pretty straight forward - it's the time when the task should be done.  But what is DTSTART?  Say, some bureaucracy work needs to be done "this year" - DUE should obviously be set to 1st of January at 00:00.
 
-Different task list implementations may behave differently, most of them is probably only concerned with the due date.
+As of 2015 my opinion was that DTSTART is the earliest time you expect to start working with the task, or maybe the earliest time it's possible to start.  Say, we plan to sit down and do bureaucraziness the 15th of December.
 
-According to the RFC, either due or duration should be given, never both of them.  A dtstart and a duration should be considered as equivalent with a dtstart and a due date (where due = dtstart + duration).  I find that a bit sad, I'd like to use dtstart as the time I expect to start working with a task, duration as the time I expect to actually use on the task, and due as the date when the task really should be completed.  Calendar-cli does not support the duration field as of today.
+Passing DTSTART doesn't mean you need to drop everything else and start working on the task immediately.  My idea was to restrict the todo-list to tasks where the DTSTART was already passed ... and then one could postpone the dtstart just to unclutter the todo-list.  However, I think it is more desirable to use the DURATION field for estimations of how long time the task will take.  Now, this bureaucraziness may be estimated to three hours of work.  That means DTSTART should be set to 21:00 at New Years eve.  Now, that's just silly!  But yeah, the DTSTART has a meaning: that's the time you need to drop everything else if you didn't do the task yet.
 
-Sometimes one gets overwhelmed, maybe one gets a week or two behind the schedule due to external circumstances.  calendar-cli supports operations like "add one week to all tasks that haven't been done yet".  As of 2015 the default due date was one week in the future.  It has been changed to one year in the future.  Probably the best practice is to keep the due date unset unless the task has a hard due date.
+I have some more thoughts on project management in the other document, [NEXT_LEVEL](NEXT_LEVEL.md).
 
-It's also possible to set a priority field.
+Priority
+--------
+
+The RFC defines priority as a number between 0 and 10.
 
-The default sorting is:
+0 means the priority is undefined, 1-4 means the priority is "high", 5 that it's "medium high" and 6-10 means the priority is "low".
 
-* overdue tasks at the very top.
-* tasks that have passed dtstart above tasks that haven't passed dtstart
-* within each group, sort by due-date
-* if equal due-date, sort by dtstart
-* if equal due-date and dtstart, sort by priority
+Should tasks be done in the order of their priority?  Probably not, as there is also the DUE-date to consider.  I do have some ideas on how to sort and organize tasks in the [NEXT_LEVEL](NEXT_LEVEL.md) document.  To follow the thoughts there, let priority be defined as such:
 
-My usage pattern so far:
+1: The DUE timestamp MUST be met, come hell or high water.
+2: The DUE timestamp SHOULD be met, if we lose it the task becomes irrelevant.
+3: The DUE timestamp SHOULD be met, but worst case we can probably procrastinate it, perhaps we can apply for an extended deadline.
+4: The deadline SHOULD NOT be pushed too much
+5: If the deadline approaches and we have higher-priority tasks that needs to be done, then this task can be procrastinated.
+6: The DUE is advisory only and expected to be pushed - but it would be nice if the task gets done within reasonable time.
+7-9: Low-priority task, it would be nice if the task gets done at all ... but the DUE is overly optimistic and expected to be pushed several times.
 
-* Skip using the priority field.
-* Try to handle or postpone or delete tasks that are overdue immediately, we shouldn't have any overdue tasks (oups, this didn't work out very well, as the default due date was 7 days in the future).
-* On a daily basis, go through all uncategorized tasks.  All tasks should have at least one category set.  I typically do this while sitting on the metro in the morning.
-* On a daily basis, look through all tasks that have passed the dtstart timestamp.  I'm considering when/how to do those tasks and (re)consider what is most urgent.  It's important to keep this list short (or it gets unwieldy, demotivating, etc), so I procrastinate everything I know I won't get done during the next few days.  I move the dtstart  timestamp to some future date - depending on my capacity, the importance of the tasks, etc, I add either some few days, weeks, months or years to it.
-* Whenever I think of something that needs to be done, I add it on the task list, immediately, from the telephone.
-* I've been using the timestamps to try to prioritize the tasks.
-* Whenever I'm at some specific location or doing some specific work, I'll filter the task list by category, often including tasks with future dtstart.
+Recommendation: split ut tasks
+------------------------------
 
-I believe my approach of using timestamps rather than the priority field makes some sense; by looking through the "task list for this week" on a daily basis, and adding some weeks to those I know I won't be able to do anytime soon, the task list is always being circulated, there are no tasks that really gets forgotten.
+Tasks that takes more than some few hours ought to be split up into several subtasks.
 
-Task lists tend to always grow, at some point it's important to realize ... "Those tasks are just not important enough ... I'll probably never get done with those tasks", and simply delete them.  I'm not so good at that, my alternative approach is to set a due-date and dtstart in the far future.  I remember back in 2005, 2008 was the year I was going to get things done.  Hm, didn't happen. :-)
+To increase the probability that a high-priority task is done before the DUE, it may also be smart to split it up into subtasks/dependencies with lower priority but due dates set according to when one is expecting to get done with them.
\ No newline at end of file
diff --git a/USER_GUIDE.md b/USER_GUIDE.md
index fcfc10a..196824c 100644
--- a/USER_GUIDE.md
+++ b/USER_GUIDE.md
@@ -24,6 +24,14 @@ kal command subcommand --help
 * add - for adding things to the calendar(s)
 * select - for selecting, viewing, editing and deleting things from the calendar(s).
 
+## Convenience commands
+
+Those commands are made mostly for making `kal` more convenient to use.  Many of the commands are optimized for the work flows of the primary author.  I may eventually decide to "hide" the more obscure commands from the `--help` overview.  (TODO: figure out if subcommands can be grouped in the help printed by click)
+
+* fix_tasks_interactive - go through all tasks that are missing categories, due date, priority, duration (technically, DTSTART) and ask interactively for values.
+* agenda - list up some of the upcoming events plus some of the upcoming tasks
+* interactive_config - (TODO: NOT IMPLEMENTED YET).  This one is not used by the primary author and is probably under-tested.  Its primary intention is to make it easy for others to use the tool.
+
 ## Global options
 
 The global options are for setting the connection parameters to the server and choosing what calendar(s) to operate at.  Connection parameters may be typed in directly:
diff --git a/calendar_cli/cal.py b/calendar_cli/cal.py
index e1004e9..0ed3502 100755
--- a/calendar_cli/cal.py
+++ b/calendar_cli/cal.py
@@ -87,8 +87,21 @@ def parse_dt(input, return_type=None):
 
 def parse_add_dur(dt, dur):
     """
-    duration may be something on the format 1s (one second), 3m (three minutes, not months), 3.5h, 1y1w, etc
-    or a ISO8601 duration (TODO: not supported yet).  Return the dt plus duration
+    duration may be something like this:
+      * 1s (one second)
+      * 3m (three minutes, not months
+      * 3.5h
+      * 1y1w
+    
+    It may also be a ISO8601 duration
+
+    Returns the dt plus duration.
+
+    If no dt is given, return the duration.
+
+    TODO: months not supported yet
+    TODO: return of delta in years not supported yet
+    TODO: ISO8601 duration not supported yet
     """
     time_units = {
         's': 1, 'm': 60, 'h': 3600,
@@ -101,10 +114,13 @@ def parse_add_dur(dt, dur):
         u = rx.group(2)
         dur = rx.group(3)
         if u=='y':
-            dt = datetime.datetime.combine(datetime.date(dt.year+i, dt.month, dt.day), dt.time())
+            return datetime.datetime.combine(datetime.date(dt.year+i, dt.month, dt.day), dt.time())
         else:
-            dt = dt + datetime.timedelta(0, i*time_units[u])
-    return dt
+            dur = datetime.timedelta(0, i*time_units[u])
+            if dt:
+                return dt + dur
+            else:
+                return dur
    
 
 ## TODO ... (and should be moved somewhere else?)
@@ -241,7 +257,10 @@ def cli(ctx, **kwargs):
 
 @cli.command()
 @click.pass_context
-def interactive_config(ctx):
+def i_update_config(ctx):
+    """
+    Edit the config file interactively
+    """
     raise NotImplementedError()
 
 @cli.command()
@@ -271,7 +290,7 @@ def _set_attr_options_(func, verb, desc=""):
         verb = ""
     if verb == 'no-':
         for foo in attr_txt_one + attr_txt_many + attr_time + attr_int:
-            func = click.option(f"--{verb}{foo}/--defined-{foo}", default=None, help=f"{desc} ical attribute {foo}")(func)
+            func = click.option(f"--{verb}{foo}/--has-{foo}", default=None, help=f"{desc} ical attribute {foo}")(func)
     else:
         if verb == 'set-':
             attr__one = attr_txt_one + attr_time + attr_int
@@ -309,8 +328,18 @@ def _set_attr_options(verb="", desc=""):
 @click.option('--offset', help='SKip the first objects', type=int)
 @click.pass_context
 def select(*largs, **kwargs):
-    """
-    select/search/filter tasks/events, for listing/editing/deleting, etc
+    """Search command, allows listing, editing, etc
+
+    This command is intended to be used every time one is to
+    select/filter/search for one or more events/tasks/journals.  It
+    offers a simple templating language built on top of python
+    string.format for sorting and listing.  It offers several
+    subcommands for doing things on the objects found.
+
+    The command is powerful and complex, but may also be non-trivial
+    in usage - hence there are some convenience-commands built for
+    allowing the common use-cases to be done in easier ways (like
+    agenda and fix-tasks-interactive)
     """
     return _select(*largs, **kwargs)
 
@@ -424,17 +453,45 @@ def _select(ctx, all=None, uid=[], abort_on_missing_uid=None, sort_key=[], skip_
     if limit is not None:
         ctx.obj['objs'] = ctx.obj['objs'][0:limit]
 
+    ## some sanity checks
+    for obj in ctx.obj['objs']:
+        comp = obj.icalendar_component
+        dtstart = comp.get('dtstart')
+        dtend = comp.get('dtend') or comp.get('due')
+        if dtstart and dtend and dtstart.dt > dtend.dt:
+            logging.error(f"task with uuid {comp['uid']} as dtstart after dtend/due")
+
+@select.command()
+@click.pass_context
+def list_categories(ctx):
+    """
+    List all categories used in the selection
+    """
+    cats = _cats(ctx)
+    for c in cats:
+        click.echo(c)
+
+def _cats(ctx):
+    categories = set()
+    for obj in ctx.obj['objs']:
+        cats = obj.icalendar_component.get('categories')
+        if cats:
+            categories.update(cats.cats)
+    return categories
+
+list_type = list
+
 @select.command()
 @click.option('--ics/--no-ics', default=False, help="Output in ics format")
-@click.option('--template', default="{DUE.dt:?{DTSTART.dt:?(date missing)?}?%F %H:%M:%S}: {SUMMARY:?{DESCRIPTION:?(no summary given)?}?}")
+@click.option('--template', default="{DTSTART.dt:?{DUE.dt:?(date missing)?}?%F %H:%M:%S}: {SUMMARY:?{DESCRIPTION:?(no summary given)?}?}")
 @click.pass_context
 def list(ctx, ics, template):
     """
-    print out a list of tasks/events/journals
+    Print out a list of tasks/events/journals
     """
     return _list(ctx, ics, template)
 
-def _list(ctx, ics=False, template="{DUE.dt:?{DTSTART.dt:?(date missing)?}?%F %H:%M:%S}: {SUMMARY:?{DESCRIPTION:?(no summary given)?}?}"):
+def _list(ctx, ics=False, template="{DTSTART.dt:?{DUE.dt:?(date missing)?}?%F %H:%M:%S}: {SUMMARY:?{DESCRIPTION:?(no summary given)?}?}"):
     """
     Actual implementation of list
     """
@@ -460,6 +517,11 @@ def _list(ctx, ics=False, template="{DUE.dt:?{DTSTART.dt:?(date missing)?}?%F %H
 @select.command()
 @click.pass_context
 def print_uid(ctx):
+    """
+    Convenience command, prints UID of first item
+
+    This can also be achieved by using select with template and limit
+    """
     click.echo(ctx.obj['objs'][0].icalendar_component['UID'])
 
 @select.command()
@@ -467,7 +529,7 @@ def print_uid(ctx):
 @click.pass_context
 def delete(ctx, multi_delete, **kwargs):
     """
-    delete the selected item(s)
+    Delete the selected item(s)
     """
     objs = ctx.obj['objs']
     if multi_delete is None and len(objs)>1:
@@ -498,23 +560,30 @@ def _edit(ctx, add_category=None, complete=None, complete_recurrence_mode='safe'
         complete_recurrence_mode = kwargs.pop('recurrence_mode')
     _process_set_args(ctx, kwargs)
     for obj in ctx.obj['objs']:
-        ie = obj.icalendar_component
+        component = obj.icalendar_component
         if kwargs.get('pdb'):
+            click.echo("icalendar component available as component")
+            click.echo("caldav object available as obj")
+            click.echo("do the necessary changes and press c to continue normal code execution")
+            click.echo("happy hacking")
             import pdb; pdb.set_trace()
         for arg in ctx.obj['set_args']:
             if arg in ('child', 'parent'):
                 obj.set_relation(arg, ctx.obj['set_args'][arg])
+            elif arg == 'duration':
+                duration = parse_add_dur(dt=None, dur=ctx.obj['set_args'][arg])
+                obj.set_duration(duration)
             else:
-                if arg in ie:
-                    ie.pop(arg)
-                ie.add(arg, ctx.obj['set_args'][arg])
+                if arg in component:
+                    component.pop(arg)
+                component.add(arg, ctx.obj['set_args'][arg])
         if add_category:
-            if 'categories' in ie:
-                cats = ie.pop('categories').cats
+            if 'categories' in component:
+                cats = component.pop('categories').cats
             else:
                 cats = []
             cats.extend(add_category)
-            ie.add('categories', cats)
+            component.add('categories', cats)
         if complete:
             obj.complete(handle_rrule=complete_recurrence_mode, rrule_mode=complete_recurrence_mode)
         elif complete is False:
@@ -527,7 +596,11 @@ def _edit(ctx, add_category=None, complete=None, complete_recurrence_mode='safe'
 @click.option('--recurrence-mode', default='safe', help="Completion of recurrent tasks, mode to use - can be 'safe', 'thisandfuture' or '' (see caldav library for details)")
 def complete(ctx, **kwargs):
     """
-    Mark tasks as completed (alias for edit --complete)
+    Convenience command, mark tasks as completed
+
+    The same result can be obtained by running this subcommand:
+
+      `edit --complete`
     """
     return _edit(ctx, complete=True, **kwargs)
 
@@ -543,18 +616,108 @@ def sum_hours(ctx, **kwargs):
 
 @cli.command()
 @click.pass_context
+def i_set_task_attribs(ctx):
+    """Interactively populate missing attributes to tasks
+
+    Convenience method for tobixen-style task management.  Assumes
+    that all tasks ought to have categories, a due date, a priority
+    and a duration (estimated minimum time to do the task) set and ask
+    for those if it's missing.
+
+    See also USER_GUIDE.md, TASK_MANAGEMENT.md and NEXT_LEVEL.md
+    """
+    ## Tasks missing a category
+    LIMIT = 16
+
+    def _set_something(something, help_text, default=None):
+        cond = {f"no_{something}": True}
+        if something == 'duration':
+            cond['no_dtstart'] = True
+        _select(ctx=ctx, todo=True, limit=LIMIT, sort_key=['{DTSTART.dt:?{DUE.dt:?(0000)?}?%F %H:%M:%S}', '{PRIORITY:?0?}'], **cond)
+        objs = ctx.obj['objs']
+        if objs:
+            num = len(objs)
+            if num == LIMIT:
+                num = f"{LIMIT} or more"
+            click.echo(f"There are {num} tasks with no {something} set.")
+            if something == 'category':
+                _select(ctx=ctx, todo=True)
+                cats = list_type(_cats(ctx))
+                cats.sort()
+                click.echo("List of existing categories in use (if any):")
+                click.echo("\n".join(cats))
+            click.echo(f"For each task, {help_text}")
+            for obj in objs:
+                comp = obj.icalendar_component
+                summary = comp.get('summary') or comp.get('description') or comp.get('uid')
+                value = click.prompt(summary)
+                if not value and default:
+                    value = default
+                if something == 'category':
+                    comp.add('categories', value.split(','))
+                elif something == 'due':
+                    obj.set_due(parse_dt(value), move_dtstart=True)
+                elif something == 'duration':
+                    obj.set_duration(parse_add_dur(None, value), movable_attr='DTSTART')
+                else:
+                    comp.add(something, value)
+                obj.save()
+            click.echo()
+
+    ## Tasks missing categories
+    _set_something('category', "enter a comma-separated list of categories to be added")
+
+    ## Tasks missing a due date
+    _set_something('due', "enter the due date (default +2d)", default="+2d")
+
+    ## Tasks missing a priority date
+    message="""Enter the priority - a number between 0 and 9.
+
+The RFC says that 0 is undefined, 1 is highest and 9 is lowest.
+
+TASK_MANAGEMENT.md suggests the following:
+
+1: The DUE timestamp MUST be met, come hell or high water.
+2: The DUE timestamp SHOULD be met, if we lose it the task becomes irrelevant.
+3: The DUE timestamp SHOULD be met, but worst case we can probably procrastinate it, perhaps we can apply for an extended deadline.
+4: The deadline SHOULD NOT be pushed too much
+5: If the deadline approaches and we have higher-priority tasks that needs to be done, then this task can be procrastinated.
+6: The DUE is advisory only and expected to be pushed - but it would be nice if the task gets done within reasonable time.
+7-9: Low-priority task, it would be nice if the task gets done at all ... but the DUE is overly optimistic and expected to be pushed several times.
+"""
+    
+    _set_something('priority', message, default="5")
+
+    ## Tasks missing a duration
+    message="""Enter the DURATION (i.e. 5h or 2d)
+
+TASK_MANAGEMENT.md suggests this to be the estimated efficient work time
+needed to complete the task.
+
+(According to the RFC, DURATION cannot be combined with DUE, meaning that we
+actually will be setting DTSTART and not DURATION)"""
+
+    _set_something('duration', message)
+
+@cli.command()
+@click.pass_context
 def agenda(ctx):
     """
-    Prints an agenda (alias for select --event --start=now --end=in 32 days --limit=16 list)
-    plus a task list (alias for select --todo --sort '{DUE.dt:?{DTSTART.dt:?(0000)?}?%F %H:%M:%S}' --sort '{PRIORITY:?0}' --limit=16 list)
+    Convenience command, prints an agenda
+
+    This command is slightly redundant, same results may be obtained by running those two commands in series:
+    
+      `select --event --start=now --end=in 32 days --limit=16 list`
+    
+      `select --todo --sort '{DTSTART.dt:?{DUE.dt:?(0000)?}?%F %H:%M:%S}' --sort '{PRIORITY:?0}' --limit=16 list`
 
     agenda is for convenience only and takes no options or parameters.
-    Use the select command for advanced usage.
+    Use the select command for advanced usage.  See also USAGE.md.
     """
     start = datetime.datetime.now()
     _select(ctx=ctx, start=start, event=True, end='+30d', limit=16, sort_key=['DTSTART', 'get_duration()'])
     objs = ctx.obj['objs']
-    _select(ctx=ctx, start=start, todo=True, end='+30d', limit=16, sort_key=['{DUE.dt:?{DTSTART.dt:?(0000)?}?%F %H:%M:%S}', '{PRIORITY:?0?}'])
+    _select(ctx=ctx, start=start, todo=True, end='+30d', limit=16, sort_key=['{DTSTART.dt:?{DUE.dt:?(0000)?}?%F %H:%M:%S}', '{PRIORITY:?0?}'])
     ctx.obj['objs'] = objs + ["======"] + ctx.obj['objs']
     return _list(ctx)
 
@@ -615,9 +778,7 @@ def _process_set_args(ctx, kwargs):
         elif x.startswith('set_'):
             ctx.obj['set_args'][x[4:]] = kwargs[x]
     for arg in ctx.obj['set_args']:
-        if arg == 'duration':
-            raise NotImplementedError
-        if arg in attr_time:
+        if arg in attr_time and arg != 'duration':
             ctx.obj['set_args'][arg] = parse_dt(ctx.obj['set_args'][arg])
 
     if 'summary' in kwargs: