Searchable Tags in td

I'm starting to really like writing console apps in Python. To add a feature takes no time at all since there is little UI to deal with. Over the last hour I've implemented tagging for your tasks in td. Tags for searching and grouping items seem to be pretty popular in current applications, so I thought why not a todo list manager? I think I like this better than the projects I implemented a couple of weeks ago. The projects stuff is of course still available and fully functional, I just don't think I'll be using it as much anymore because the new tags are much handier. In fact, you can still use both at the same time. Here is how the new tags command works...

When you create a new task, a group of tags can be associated with it by including a space separated list, enclosed in square brackets, in the task summary.

$ td add [tag1 tag2 tag3]This is a new task.

Now, if I want to see what tags are being used:

$ td tags
--> Available tags:
--> tag1(1) tag2(1) tag3(1)

Issuing the new tags command with no arguments lists all tags, and the number of tasks with that tag. If you want to see a list of tasks by tag:

$ td tags tag1
-->  #   P H  Created    Description
-->  1        12/20/06   This is a new task.

Maybe you want to see a list of tags that are associated with a specific task?

$ td tags 1
--> [tag1 tag2 tag3]

Want to remove the "tag2" tag from that task? No problem...

$ td tags 1 -tag2
$ td tags 1
--> [tag1 tag3]

Let's remove "tag3" now, and add "newtag" to this task:

$ td tags 1 -tag3 +newtag
$ td tags 1
--> [tag1 newtag]

You can still use the replace command to replace all of them at once.

$ td replace 1 [foo bar]This is a new task.
$ td tags 1
--> [foo bar]

And of course you can easily combine hidden status, tags, and projects all at once.

$ td add .[tag1 tag2] p:MyProject This is yet another new task.
$ td tags tag1
-->  #   P H  Created    Description
-->  2     *  12/20/06   This is yet another new task.
$ td project MyProject
-->  #   P H  Created    Description
-->  2     *  12/20/06   This is yet another new task.

Notice the "H" column in that output indicates whether the task in the list is hidden or not. That column was recently added to both the tags and the project output. These changes are committed and pushed to the bazaar repository that I spoke of in my last post. Don't forget to email me the bugs!

Now Using Bazaar for a VCS

In order to hang with the cool crowd, I've decided to fall for the hype and jump ship (at least for now) to a distributed version control system. The one I've chosen is bazaar, mostly because it's written purely in Python. I like playing with new things from time to time, and bazaar commands are a pretty smooth transition from their Subversion equivalents.

So, in order to get the latest and greatest unreleased code for td or vim-blogger, you can use bazaar and grab the source from here and here respectively.

If need a quick learning reference, have a look at the bazaar tutorial. It seems the User Manual link on the website points to the tutorial as well.. Hmm. To find out more about the program, the official website can be found here.

Vim-BloggerBeta Plugin Release

It turns out I'm not really doing any more modifications to this plugin code because it is working fine for me. I decided then, instead of just sitting on it maybe I should share it! So here it is... Keep in mind that this no doubt a little rough around the edges as far as usability goes. I'll do my best to describe the plugin here.

Pre-requisites

First of all, this plugin requires Python support to be compiled into Vim. As in, if you do a :version in Vim and do not see +python in the output, this plugin will not work.

Also, you will need to provide some information for the plugin in the form of three variable settings. The plugin needs to know your blog URL, your Gmail login, and your Gmail password. The variable names for these are g:Blog_URI, g:Gmail_Account, and g:Gmail_Password respectively. Given that some of these variables contain sensitive information, you probably want to take some steps to keep them somewhat private. I often put my ~/.vimrc online for others to read, so putting them in there is a bad idea. Instead, I created a ~/vim/plugin/creds.vim file, made it readable by my user only, and stuck the info in there. Keep in mind, however, that these are global variables and as such, will be available to all scripts that you run. Also, someone could quite easily get those values by typing :echo g:Gmail_Password in your vim session for example. If you have some ideas about how to keep this stuff more secure I'd be interested in hearing about them. For now, however, you've been warned :).

Installation

The plugin comes in the form of a tarball. Inside this tarball are five files.

• plugin/blogger.vim: This script provides a single command called :Blog that simply opens a temporary blogger file which in turn sources the blogger.vim filetype plugin.
• ftplugin/blogger/blogger.vim: This is the filetype plugin that contains the rest of the plugin's functionality.
• syntax/blogger.vim: The included syntax file provides highlighting to buffers with the filetype blogger. It is essentially the Markdown syntax file found here with a few vim-blogger specific additions.
• ftplugin/blogger/markdown.py: This is the Python module found here that the plugin uses to convert Markdown to HTML.
• ftplugin/blogger/html2text.py: The Python module found here which the plugin uses to convert HTML back to Markdown.

To install, you just need to untar this tarball inside your ~/.vim/ directory. All should fall into the appropriate locations. Next, you probably want to make Vim set the filetype to blogger whenever it opens an appropriate file. To do this, put this line in your ~/.vim/filetype.vim file:

autocmd! BufRead,BufNewFile     *.blogger   setf blogger

This will set the filetype accordingly whenever you open a file with the blogger extension, hence sourcing the ftplugin and syntax files.

General Post Form

In a Vim buffer, a post generally looks like this screenshot:

The first line in the buffer is the post subject. The last line contains the keyword @@LABELS@@ followed by a comma separated list of labels for this post. The stuff in the middle is the post body. One variation of this is if you are editing a post that was already published. In this case, the first line will be of the form @@EDIT#@@. This is put there by the plugin, and used by the plugin. You probably don't want to delete it. In this case, the post subject will be the second line in the buffer.

Usage

There are several commands included with the plugin. I'll briefly describe each of them here.

Retrieving Your Most Recent Posts

Running the :BlogIndex command will display a list of your most recent blog posts. By default, it will only display the last five to keep the list manageable, but it can take an integer argument if you would like to change that. A numbered list of posts is shown. To begin editing one of these posts, simply type the post number and press Enter. This list also contains posts that you have saved as drafts. Drafts are distinguishable by the word **DRAFT** beside the post subject.

Retrieving Your Posts By Label

If you would like to see a list of blog posts by label, run the :BlogQuery command with a comma separated list of labels that you would like to query for. Again, a numbered list is returned. To edit a specific post, type the appropriate number and press Enter.

Publishing or Updating a Post

The :BlogPost command can be used to either publish a new post or to update an existing post. If the current post is marked as being a draft, then the draft status is removed and the post is published. Upon successfully issuing this command, it will appear in the list displayed by the :BlogIndex command.

Saving a Post as Draft

Just like the :BlogPost command described above, the :BlogDraft command sends your post to the Blogger server, only this time it is marked as a draft. The post will not appear in your blog, but will appear in the plugin's index (displayed with the :BlogIndex command) as a **DRAFT** post.

Deleting a Post

To delete a post, run the :BlogIndex command to get a list of posts, choose the post from the list as if you were going to edit it, and issue the :BlogDelete command. The plugin will ask the user for confirmation, and upon responding with "yes" (nothing else will do), the post is deleted from your blog.

Using or Not Using Markdown

One of the things I like about this plugin is that I can use Markdown syntax to write my posts. This text is then converted by the plugin to HTML, and sent to the Blogger servers as a post. If I then want to edit that same post, the HTML code is retrieved from the server, converted back to Markdown, and made available for me to edit. Some people, however, will inevitably prefer to just compose their posts in straight HTML. For these people, there is a global variable called g:Blog_Use_Markdown which enables this portion of the plugin. To reduce confusion for the people who do not know about this feature of the plugin I've made this variable default to False. Therefore, if you want to use the Markdown stuff, put let g:Blog_Use_Markdown = 1 in your ~/.vimrc file.

Conclusions

Keep in mind that this plugin was written to scratch a particular itch of mine that was caused by slow web interfaces with a cruddy editor in a slow web browser on a slow laptop. I've been using it to make all of the posts that you see on this blog. However, it may or may not do what you expect when you expect it. Hopefully there are few actual errors, but I'm sure there will be some as I've not fully tested the plugin under every scenario. If you do find a bug or need help getting it to work, email me at my Gmail account, username "dcraven". General comments are welcomed here on the blog.

Install Packages Available for td

Jason has graciously created a Gentoo ebuild for td and made it available for download here. It looks like it even installs the td.vim file in the appropriate spot. Neato. Thanks for that!

This inspired me to make a deb file so that it's installable via apt on Ubuntu systems. The deb file for version 0.3 is available from here. I've only tested this deb by installing it once on my own system. I'm not a package building expert by any means though. Let me know if there are problems with it. Lintian doesn't seem to have any issues at all with it. There's a first.

Version 0.3 of td Released

You thought this day would never come. The highly anticipated 0.3 release of the td command line task list manager is finally ready for public release! There have been a few fixes and additions to the feature list. I keep adding features here and there as I find things lacking during daily use. Here is a list of what's been added since I rolled the 0.2 tarball:

• Support for 'hidden' tasks has been added.
• Tasks can be grouped by and assigned to projects.
• You can keep details or notes associated with your tasks.
• Support for dark or light console backgrounds.
• Choose between terse or more verbose "feedback" from commands.
• Man pages are added for both td(1) itself, and the new tdrc(5) configuration file.

I think that's pretty much it. To be honest I can't really remember what was in 0.2 to begin with. The tarball can be downloaded from here, and the original post has been updated to reflect the new features.

Hopefully my server can handle the huge amounts of downloads expected over the next few days. If not, I may need to seek hosting elsewhere ;). If you have any bug reports or feature requests, please send them to me at my gmail address with username "dcraven". General comments can either be emailed to me or left in the comments section of this blog. Either way, they are appreciated.

td Gets Project Stuff

Borrowing both the idea and the syntax from here, I have added similar project functionality to td. Now you can assign a task to a project by including a word in the description that is prefixed with 'p:'. For example, the following command will add a task to the "Office" project.

$ td add p:Office Phone that guy about the thing.

The difference between the implementation in td and the one at todotxt.com is that, in the above example, the task will appear in any displayed list as "Phone that guy about the thing.". That is, the "p:Office" part is removed. I think this keeps the descriptions a little cleaner.

Of course, this demanded an new command be added to the arsenal. That new command is proj. If the command td proj is given at the command-line, then a list of projects containing tasks is displayed. If however you provide a specific project like td proj Office, then a list of incomplete tasks belonging to that project is returned. The returned list is numbered as if a list of all incomplete tasks were requested, however, the tasks not belonging to the specified project are filtered out. To remove a task from a project, use the rep command, and exclude the prefixed word from the new description.

There is no tarball of td that includes this stuff just yet, so if you want to try it, you can check it out of the SVN repository located at http://arker.homelinux.org/td/ for now. Just like a good repository should, it also includes some glaring bugs that were found in the 0.2 tarball :).

UPDATE (Dec 14, 2006): I've started to use bazaar as a version control system for the td source code now. I've not used it before, so it sounded like fun. The bazaar branch can be found at http://arker.homelinux.org/~dcraven/bazaar/td/. I hope it works.

In the case where find a bug, or even have a feature request, email a bug report to me at dcraven at gmail dot com.

td - A Command-line Todo List Manager

UPDATE: As of version 0.4, td depends on python-dateutils for date manipulation functions. Please make sure you have this installed prior to using td.

Introduction

Over a year ago, a friend of mine and I began tinkering with a little todo list program as we played with Python. We just wanted something simple and fast that could be used from the bash prompt without the fuss of a GUI. The result was a little script that we called td. For the past year, not much changed with it. I did a bit of work on it while learning how to use gnome-vfs, and added the ability to use remote data transparently. A couple of days ago, I began playing with it again. I added a few new and useful features, and decided to make it available to everyone in case somebody out there wanted such a tool. I'll outline the features of the program here.

Installation

The latest tarball of td can be downloaded from here, and is installable using the GNU autotools. In summary, navigate to the directory that has the downloaded tarball and type the following commands.

$ tar xvzf td-0.4.tar.gz
$ cd td-0.4/
$ ./configure
$ make
$ make install

You should now be able to run the program by typing td at the command line.

Features

The td script has the ability to add, delete, complete, prioritize, and archive tasks. This section will document each feature one at a time.

Displaying Tasks

Just running the command td with no arguments will display a list of outstanding tasks, sorted by descending priority. The output looks like this:

The output contains the item or task number, the priority of the task, the creation date, and the task summary itself. The task number shown in this list is useful for performing various other commands. The last line of output displays the number of shown tasks, and the number hidden (not shown).

Adding a New Task

The add command adds a new task to the list. For example:

$ td add Remind Jason to schedule his proposal.

This procedure is limited only by what is acceptable at the bash prompt. That is, things like quotes need to be escaped for example.

$ td add These \"quotes\" are escaped.

Deleting a Task

Deleting tasks can be done by specifying the task number to delete. For example,

$ td del 2

The above command will obviously delete task number 2 from the task list. Once deleted, tasks can no longer be accessed.

Deferring a Task

Sometimes you want to add a task to your list now, but don't want it to clutter up your task list until some date in the future. To do this, you would use the defer command. This command takes several types of parameters to determine the date to defer the task to. For example, the following command will defer task number 4 one week into the future.

$ td def 4 1w

Similar short forms exist for days (d), months (m), and years (y). The defer command also uses the python-dateutils to parse dates described by natural language. For example the following command will defer task number 4 to August 5th of this year:

$ td def 4 aug 5

If no further arguments are given to the defer command, a list of deferred tasks are displayed with their creation date, and the date they have been deferred to. Deferred tasks will not appear in the regular task list until their defer date is reached.

Undeferring a Task

If a task has been deferred to a future date, it can be made current again by using the undefer command. Obtain the task number by running td with the defer command without arguments. The following command will make the deferred task number 2 current again, hence making it display in the regular list of tasks.

$ td undef 2

Task Details

Some tasks require further details to clarify it further, or maybe just a place to store additional braindumps that you'd like to jot down. Details or notes can be added to a task using the details command. This command will cause your favourite editor to open, allowing you to type your notes. This additional data will be stored with the task data, and can be accessed by running the details command again on the same task. The following command can be used to added details to task number 4 in the list.

$ td details 4

Display Hidden Tasks

To display all incomplete tasks including the tasks marked hidden, use the hid command.

$ td hid

This will show all incomplete tasks in the list with an additional column marked "H". Each task that is marked hidden in the list will contain an asterisk in this column.

Prioritizing a Task

Prioritizing tasks can be done using the pri command. Entering the following command will assign task number 3 to the highest priority.

$ td pri 3 A

Priorities can be any single letter, and the display will be sorted by this key in descending order. Priorities of A, B, and C have distinct colours in the display output. Other priorities are just dark grey. By default, new tasks are given no priority at all. To remove the priority that was previously assigned to a task, run the same command, but exclude the priority.

$ td pri 3

This command will remove the priority from task 3.

Assigning Tasks to a Project

Tasks can be grouped into "projects" in td. To add a task to a specific project, you must include the project's name, prefixed with a "p:" somewhere in the tasks summary. The following command adds a task to the Office project.

$ td add p:Office Call Steve to schedule a meeting.

If you would like to see a list of projects that have outstanding tasks associated with them, you can use the proj command with no parameters. This will provide a comma separated list of projects. To display a list of tasks assigned to a specific project, run the same command with the project as an argument. This command displays all tasks under the Office project.

$ td proj Office

Already existing tasks may be added to a project using the rep (replace) command with the "p:" prefixed project in the new summary.

Assigning Tags to a Task

The td program supports "tagging" tasks with searchable tags. This can either be done when adding a new task, or after a task has already been created. To create a task with a set of tags, include the space separated tags in the task summary enclosed in square brackets. The following command creates a new task with the tags "work" and "python":

$ td add [work python]Script the emulator to run tests over night.

If you want to add tags to an already created task, you can use the tags command. To do this, append the tag with a '+' sign prepended to it. Removing tags from a task can be performed by appending a tag with a '-' sign. For example, the following command adds the tag "work", and removes the tag "python" from task number 5:

$ td tags 5 +work -python

To view a list of tags that have been assigned to any task, use the tags command with no arguments. This will display all tags as well as the number of tasks that have that tag assigned to them. Also, to see a list of tasks with a give tag or set of tags, use the tags command with the tags of interest as an argument. For example, the following command will display a list of incomplete tasks that have been tagged with "work" and "python":

$ td tags work python

To view a list of tasks that have no tags assigned to them, use the notag command with no arguments.

Completing a Task

Tasks can be marked as complete by issuing the do command. Once marked complete, tasks will no longer appear in the normal display list, and a completion date is assigned to them. The following command will mark task number 2 as being completed.

$ td do 2

Viewing Completed Tasks

As mentioned, once marked complete a task will no longer be listed in the normal task list display. To view all completed tasks, issue the done command like this:

$ td done

This will display a list containing a task number, creation date, completion date, and task summary. The displayed task number will be useful for performing various commands on completed tasks.

Marking Completed Tasks as Incomplete

If you have marked a task as complete by accident, or otherwise decide that a completed task should be reopened, you may issue the undo command.

$ td undo 2

The above command will mark the completed task number 2 as incomplete. The task number is the one displayed in the list provided by the done command. This task will now appear in the normal listing, and has no assigned completion date.

Replacing a Task

You may modify the summary of a given task by using the rep command. When given a task number, this simply replaces the summary of that task with the new one provided.

$ td rep 3 Fix a spelling mistake in this task.

Searching for Tasks

Tasks can be filtered by providing search patterns to the filt command. By giving a series of patterns, only those tasks who's summary match all patterns will be displayed. The resulting list will be numbered as if the full list was being displayed, but the tasks which do not match the patterns will be "filtered". This operation uses a smartcase search, which means if any of the given patterns contain at least one upper case letter, then the search will be case sensitive. If none of the letters in the pattern are upper case, then the search will be case insensitive. See the following examples.

$ td filt case insensitive
$ td filt cAse sEnsItiVe

The filt command can take as many patterns as you wish, in any order. Only the tasks matching all patterns will be displayed.

Archiving Tasks

Once completed, tasks may be archived in a text file in case you would like to keep them for future reference. The text file is stored in ~/.td/td_archive.txt. The file contains the summary, associated project, the creation date, and the completion date of all archived tasks. This operation permanently removes tasks from the todo list. You may archive a single task by providing its number, or all completed tasks by specifying all as an argument to the archive command. If no argument is given, td will ask you if you would like to archive all completed tasks. Answering "yes" will perform the archiving, anything else will abort. So basically, both of the following commands will work as expected.

$ td archive all
$ td archive 3

Purging Tasks

To just delete all completed tasks (unarchived) without saving them in the archive, issue the purge command. If given a task number, that task alone will be purged. If no task number is given, the user is asked for confirmation, and all complete tasks are purged.

$ td purge 3

This command will only purge task 3, without confirmation.

$ td purge

This command will prompt the user for confirmation, then purge all completed tasks. Purged tasks are lost, and cannot be recovered.

Getting Help

The td help command simply prints a message with a terse description of all the available commands, and their usage.

Specifying Your Data Source

This really only applies if you do not have gnome-vfs and its Python bindings installed. To set the data source after first run (see below), you can issue the source command. Doing this without the presence of gnome-vfs has no effect. However, you can change your data source to a remote file if you do have gnome-vfs installed. For example, the following command will make td use a remote file to store data over ssh.

$ td source sftp://your.host.com:/home/username/.td/todos.xml

From this point on you can use td as usual, and it will transparently use the data file stored at the given URI. The source command also takes the "local" argument (see below).

Without GNOME-VFS

If you do not have gnome-vfs and its Python bindings installed, the program will automatically store your todo list in the file ~/.td/todos.xml. There are no remote data capabilities without using gnome-vfs.

With GNOME-VFS

The presence of gnome-vfs and its Python bindings will be detected when the program is run. On first run, td will ask you to enter a data source. This basically just means that it needs to know where to store your data. If you enter "local" then your data will be stored locally in the file ~/.td/todos.xml. If you want to use remote data, enter any valid gnome-vfs URI to use that location. For example, the to use a data file stored in your home directory on a remote server, you could use sftp://your.host.com:/home/username/.td/todos.xml to make td use that data over ssh.

Types of Tasks

td maintains three types of tasks in its lists. This section will briefly describe each.

Incomplete Tasks

Incomplete tasks are basically any type of task that is not marked as complete. This is the default type when a task is created. These obviously represent tasks that are not yet completed, and which the user wants to appear in the default task list.

Hidden Tasks

These are a subset of incomplete tasks. They are treated the same, except the user doesn't want to see them in the default task list. Reasons for this may be to reduce clutter if for example the task is long term. Other reasons for marking a task as hidden are up to the user. Tasks are marked as hidden when, upon creation, the task summary provided begins with a period ("."), just like hidden files on the Unix file system. To mark a hidden task as no longer being hidden, use the replace command and enter a summary without the leading period. Similarly, to mark an already existing task as hidden, simply replace the current summary with one beginning with a period.

Completed Tasks

Tasks that have been marked as complete with the do command are no longer displayed in the default task list. To view these tasks, use the done command. To mark a completed task as incomplete, use the undo command. Only tasks marked as completed are available to the purge and archive commands.

The ~/.tdrc Configuration File

td now looks in $HOME/.tdrc if it exists for a few optionally configurable settings. This section will describe these settings.

The syntax used by the tdrc file is pretty typical for any configuration file. Entries are generally of the form {KEY} = {VALUE}. Pretty straightforward stuff. If the file doesn't exist, or only a subset of the options are specified in it, then sane defaults are chosen.

Background Colour

The background option informs td of the type of background on the user's xterm. If the xterm background is dark, then typically darker coloured text doesn't show up well. Setting this option to dark instructs td to use brighter colours for its output. Setting it to light instructs td to use more subtle colouring suitable for light backgrounds. The default value for this option is light.

Feedback Level

Sets the level of feedback that td should provide to the user. If the value of the feedback key is set to verbose, then positive feedback will be provided upon successful completion of commands. If set to terse then only errors will be displayed. The default value for this option is verbose.

Preferred Editor

The editor sets the preferred editor to be used when modifying the notes or details portion of a task. When deciding what editor to use, td will first check the $HOME/.tdrc file for a value. If none is set, then the value of $EDITOR is used. If still no value is found then Vim is used. This value can contain either a full path to the executable or just the executable file name provided it can be found in your $PATH. The default for this option is vim.

Extra Stuff

I like syntax colouring, and I use Vim for all of my editing. I made a very simple syntax file for editing td task details. It provides colours for the first line of the details file which indicates the project and the task summary. I also like to enter most of my notes in point form, so this syntax file also highlights lines beginning with an asterisk :). I know, I'm lame.

Anyways, this syntax file is available in the td tarball and will be installed into the $PREFIX/share/td/extras/ directory on your file system. To use it, copy it to $HOME/.vim/syntax/ and add the following line to your $HOME/.vim/filetype.vim file.

autocmd! BufRead,BufNewFile     *.td    setf td

After doing this, you should see the syntax highlighting the next time you run the details command in td.

Conclusion

If you decide to use td or at least try it out, you can probably expect a bug or two. If that is the case, or you need some clarification, feel free to email me so I can help. Although I don't expect there to be much demand for a tool like this one, if somebody besides me would find it useful, I'd like to make it work for them too.