Usability report: Pixelated Webmail Client

Pixelated’s logo.

Activity: Usability testing on an encrypted webmail client.

Takeaway: In trying to balance the expectations of new users and experienced users of encrypted email, make information about what is being encrypted available when users look for it, with greater or lesser detail depending on the use case. An extra warning is in order when there is a risk users will accidentally send messages in the clear.

About these user tests

I conducted three user tests of Pixelated, an encrypted, browser-based email client, at the Chaos Communications Camp in August 2015, and three at the Internet Freedom Festival in March 2016. Participants were primarily European and North American adults; there was one African participant. Two participants had not previously used encrypted email, while four had.

The test build of Pixelated changed between August and March, with the developers moving the fields for the email header from the bottom to the top of the message-composition area. They also responded to tickets I opened on GitHub. This report focuses on important results pertaining to features that remained the same throughout both versions, as the small sample size (three tests each) did not turn up any meaningful data about the changes.

In August, I focused the user tests on the simple task of sending an encrypted message to one of the other test accounts. In March, at ThoughtWorks’s request, I focused on having them send attachments. In both tests, I deliberately did not tell participants how Pixelated works, in order to get a sense of what they expected the software would do to protect them.

In addition to these tests, I also did an initial review of the main inbox interface with a senior usability expert, whose comments on the interface enforced feedback from user tests. Her comments on specific elements of the interface are included on the annotated screenshots.

What’s going well

• Basic navigation: Generally, users make sense of and navigate around the screen easily, even on slightly unusual features, like the compose window being at the right.
• Users quickly orient to and make sense of dialogs about attachment size limit.
• Users figure out the hamburger menu pretty easily, and use both it and the logo to expand and collapse the left-hand column.
• Saving of drafts is working seamlessly when users navigate away from their draft.
• Graphic web design and information architecture: The webpage explaining Pixelated was also beautifully easy to scan at the time of the August tests. However, by the time of this writeup it appears it has been changed to be more text-heavy, and possibly less informative for less tech-savvy users.
• One user commented on how “spacious and nice” and unconstrained by boxes the compose window feels.

What needs improvement

See the annotated screenshots of Pixelated for additional suggestions of changes to the interface, based on the expert review and user test results. Any change noted there which is not also listed below can be considered low priority.

1) Sending one message both encrypted and unencrypted, with no warnings, is a security risk — High priority

One test participant who trains journalists in digital security expressed alarm that Pixelated would allow the same message to be sent to one recipient encrypted and another recipient in the clear. “This seems completely insecure,” he said, noting that ISPs and state agencies could easily see the unencrypted message in transit. He expressed surprise that Pixelated had not stopped and notified him that it was sending a message in the clear.

Recommendations:

• Pixelated should think hard about the ramifications of sending mixed-mode messages, as there is potential for user error to put users in some situations at risk of jail, physical harm, or death.
• Further user tests should be done on whether users are noticing and responding to any mixed-mode notifications Pixelated is showing.
• User tests I have done on both Mailpile and Pixelated indicate that when you tell users an email system will encrypt and then give them the task of sending a message, they do not think about checking whether a message will send encrypted unless you stop them and ask them to look for cues about encryption.
• This is a good moment to think through use cases with personas.
• Consider building new personas which focus on the needs of users who might be sending messages with a mix of security concerns (to friends in one country who are under heavy surveillance, and another country where there is no threat, and the friend refuses to use encryption) versus uniform security concerns (messages from a country where all traffic is surveilled).
• Feel free to make use of the security-related personas we built at OpenITP.
• There are a couple of ways to deal with mixed-mode messages. All of them involve notifying the user of mixed-mode risk in some way, to make sure they do not send accidentally.
• Pixelated could follow Enigmail’s model:
• Enigmail does not permit users to hit “send” and transmit the same message encrypted and unencrypted at the same time.
• When users attempt to send a message to one user with a valid key and another user without, Enigmail prompts them to look for a key for the keyless recipient, popping up its key list.
• If the user does not select a valid key, Enigmail notifies the user that the message could not be sent because it could not to send to all recipients securely.
• It then returns them to the draft without sending.
• Users are free to send the content of the draft in the clear in one message, and copy it into another message to send it encrypted or vice versa.
• Or, Pixelated could send the message to the recipient with a key, and hold back the unencrypted version.
• After sending the encrypted version, Pixelated could show the user the queued unencrypted messages.
• On showing that queue, Pixelated should ask the user something like “Are you sure you want to send this message unencrypted? In some situations, the existence of an unencrypted version of a message could put you or your contacts at risk”
• This should have “Cancel” and “I understand” buttons. “I understand” sends the message unencrypted, “Cancel” returns them to the draft.
• There should NOT be a “don’t show this again” option for this dialog.

2) Users want mouseover or clickable info on what the locks and encryption tags mean — High priority

When confronted with an email address which did not trigger a green lock, users would mouse over or click on the locks and “encrypted” or “unencrypted” tags to try to find more information about why Pixelated would not encrypt.

Users tried to hover over or click the orange-and-green encryption-related elements above.

One participant, who does not use encrypted email, assumed the absence of a green lock when she attempted to send to a Gmail address was “because this software does not believe Gmail is secure.”

Recommendations:

• On mouseover, give users a short, descriptive phrase in simple, non-technical language about the reason why locks and encryption-related tags are appearing.
• One participant had the interesting idea to put “Remind this person to update their key” in a right-click menu on the orange lock. I’m not sure if this would work with Pixelated’s model for keys, but perhaps it could be used with individual (as opposed to institutional) keyholders?

3) Experienced users want to be able to manage keys — High priority

Participants who were regular users of encrypted mail wanted a way to change which key was being used, manage expired keys, etc. The security trainer, when trying to send an encrypted message to himself, was frustrated when Pixelated did not find a key for his address which he expected Pixelated would find on a keyserver. Again, he tried to mouse over and click on the locks in order to get more information about why Pixelated would not encrypt the message.

I am aware that Pixelated’s way of using keys is different from most encrypted mail systems. I did not inform participants in this user test about that fact. But as long as Pixelated pulls in keys from key servers, it is still making use of the traditional key infrastructure, and users who want to work with that system (experienced users like trainers and others who may recommend Pixelated to their friends — and non-experienced users who may evolve into super-users!) should be given a way to deal with missing, expired, and other problematic keys.

Recommendations:

• At the bottom of the non-technical information in a mouseover box for the encryption icons and tags, give a link with more information, where advanced users can look at more technical information about why Pixelated failed to find a usable key.
• Provide a key management option in the contact list (see below).

4) Users may want to preview the first line of emails in the inbox list — High priority

Our expert UX reviewer, a longtime user of encryption tools herself, noted that she wants to be able to preview the content of an email in order to know if a message is some sort of phishing/scam. Previewing email is thus useful for security reasons.

Reccomendation: Give users a one-line preview of the body of emails in the inbox list.

5) To: field shifts in current layout, making it hard to add more than one recipient — High priority, bug

Users struggled to add a second recipient to email, because after the first one had been entered, the field shifted slightly off the baseline and above the entered email, like this:

When users clicked near the baseline, Pixelated was unresponsive, making it look like no more recipients could be added.

Recommendation: This appears to be a bug related to the responsive layout, and should be fixed.

6) Users expect a contact list — Medium priority, feature request

Particularly when the orange unlocked-lock appeared, users went looking for a contact list to solve what they figured was a problem with a missing key. The looked in the left-hand menu, but did not find anything.

Recommendation: A contact list is clearly a feature that users expect to see in an email client. It should be prioritized in a list of features to build.

7) Search field’s purpose is not immediately obvious — Medium priority

As Pixelated is currently laid out, “compose” looks like a button that acts on the “search” field:

The search icon and label are a little subdued.

At least one user entered his To: address in the search field when trying to start a message. Our expert UX reviewer said users would be looking for a separate “search” button, and that the “magnifying glass” icon currently there would not be enough for users to orient to that field as a search field.

Recommendation: Add a more visible button that submits searches.

8) Meanings of icons in the left-hand menu are not clear — Medium priority

The “sent,” “settings,” “drafts,” “all,” and particularly the “logout” icon all caused users some confusion, ranging from surprised comments and misidentification (the paper airplane of “sent” was called an “arrow”) to unintentional logouts. The mouseover labels helped them understand better; one participant expressed gratitude for this.

Recommendations:

• Replace the “sent” icon with a more standard, demonstrative icon (a “box with arrow out” like some of the ones seen here, for example).
• Consider replacing the “log out” icon with the label “sign out.”
• At least one user commented they expected to see a sign-out option in the upper right, not the lower left.
• Compare to other, similar applications for a “settings” icon.
• Considering labeling each icon or replacing them all with labels.

9) Users expect to be able to multi-select in attachments dialog — Low priority

More than one user expressed surprise when they could not select more than one attachment at once in the attachment selection dialog.

Recommendation: Enable OS-native ways of selecting multiple attachments in the attachment dialog window, if possible.