The State of the GUI

[Adam Wilson]

On 10/24/18 5:22 PM, Neia Neutuladh wrote:
> On Wed, 24 Oct 2018 16:55:05 -0700, Adam Wilson wrote:
>> My team at MSFT did accessibility work and we had to follow all the ADA
>> guidelines, EU rules, and internal policies when creating our HTML based
>> UX. What you are describing here is not a problem with any individual
>> toolkit, but the inability or unwillingness of the individual teams
>> implementing the UX to use the accessibility tools of their respective
>> toolkits. WPF has expansive Accessibility tooling, and indeed, all of
>> the work on WPF in the .NET 4.7.x series has been on Accessibility to
>> enable MSFT to meet all applicable policies for it's own WPF based apps.
>>
>> The problems you describe with accessibility are not related to the type
>> of toolkit used. Native or Non-Native, if they don't utilize the
>> accessibility features of the toolkit, it won't work right for those
>> requiring accessible interfaces. I can build a completely inaccessible
>> UX in any toolkit. And indeed, I've built inaccessible UX's in both
>> WinForms and WPF.
> 
> It would be great if the toolkit made it simple to meet those guidelines.
> Maybe you're using a framework built on top of HTML+CSS and it helps out.
> Maybe you're using GTK+ and it does a fair bit for you. Maybe you're using
> WinForms and it falls far short, forcing you to write a lot of code.
> 
>> Specifically the goal is consistency between apps *on that platform*.
>> KDE apps look like crap on GNOME and vice versa. VS Code looks exactly
>> the same everywhere.
> 
> Consistency between platforms makes it a little easier to write
> documentation. Users get annoyed by applications that don't work like
> other applications on their platform.
> 

I've see this assertion made many times usually by developers who've 
never worked on a UX for non-technical people and always without any 
corroborating evidence.

The following is my credentials in the UX space. I took a UX design 
class from Billy Hollis, a well known (in the Microsoft space) UX 
designer. And I've study the psychology of UX design ever since then by 
reading a number of books on the subject. I've also designed a UX for 
use by non-technical people that is still in use today.

One thing I can say with confidence is that the average user wants each 
app to behave consistently regardless of where they using them. The 
human mind is an excellent pattern matching system. It turns out that as 
long as certain affordances (UX designer speak for something that 
signals deeper, possibly hidden, meaning) follow the same conventions, 
how they look (theme) on a given platform is irrelevant. An example of 
an affordance is a blinking cursor, which signals that you can type 
here. Or a drop-shadow on a button. Or a menu across the top, or the 
ubiquitous "hamburger icon" in mobile apps. As long as these affordances 
function the way their look suggests they do (by convention) then the 
specifics of how they are colored or drawn (themes, drop-shadows, etc) 
is functionally irrelevant to the human brain. This is because the brain 
creates mental "shortcuts" that map to these affordances and once the 
brain has perceived the affordance, it stops processing the particulars 
of how it is drawn.

(The exception to this rule is those who have more aggressive forms of 
Autism Spectrum Disorder. The reason is that those individuals have 
difficulty, or are even incapable of not processing. They see 
*everything*. The trouble here is that apps that an ASD person can use 
are nearly unusable to non-ASD people who are relying on the affordance 
shortcuts. So we don't do that.)

However, if I design an app that looks the same and functions the same 
across platforms (mobile, web, desktop) then the brain develops 
shortcuts for things like button color, menu ordering, etc. But the most 
important shortcut the brain can take is that of learning. For example, 
my American Express app is just a limited version of the website. I 
can't do everything the website can but what I can do works the same way 
on both, same affordances, same scheme, same layouts (within reason). I 
only have to learn the UX once.

In the UX class I took one of the first lessons was the order in how the 
  the human brain perceives the elements an image.

Colors
Shapes
Text

The first thing that registers in our brains is color. So for example, 
if we did an American Express app using the native widgets with default 
colors. Samsung's Android would be white-on-black, but Googles Pixel 
would be black-on-white. Now our brains have work harder to recognize 
the app we are using because the colors changed. So then we start 
looking at the shape on the screen, but those too have changed, Samsung 
uses different sized buttons for example, so the layout got pushed 
around. And they use different default fonts, so now the font shapes are 
different. So our brain slows down even more and starts processing the 
text on the screen and then we start to slide into our routines.

Each one of these increases the total cognitive load, and even if you 
switch between platforms many times a day, such that you get used to it, 
your brain still has to do this "context switch". It is simply 
unavoidable. You might get faster at it the more you do it, but the 
cognitive load is *always* higher. The cool psychology part of this is 
that this increased cognitive loading  produces an increase in stress 
level (cortisol). (So do square corners)

Again the exception is folks on the Autism Spectrum, they never stop 
processing, so that don't really benefit from the shortcuts. And I'd 
even agree that having an environment consistent UX for ASD folks would 
be better than a task specific UX. Based on my understanding of how ASD 
people process information.

And that is why UX teams are developling apps that look the same across 
devices, it literally de-stresses their users, which is a good thing if 
you're trying to sell something. Corporations do things that benefit 
their bottom-lines sure, but that's doesn't mean that those things are 
inherently at odds with what users want. A corporation can only make 
money from happy people. So they tend to do what makes the largest 
number of people happy.

>> Not going to disagree with HTML having a garbage box-model. But that
>> doesn't stop it from being the most widely used model out there.
> 
> Right, but if I'm choosing what tools to use, I'm going to prefer to use
> good ones instead of garbage ones.
> 

"Good" is often a matter of preference. I don't like WinForms, GTK, or 
HTML, but I do like WPF. Does that bias me, I'm sure it does, but it 
doesn't change the direction that the industry at large is headed.

>> GTK works where you can accept the limitations that system comes with.
> 
> That's suggestive of abnormal limitations, but since you haven't used
> GTK+, you presumably haven't encountered them.
> 

I'll admit that I haven't used GTK, but I've used apps that use it, and 
the thing is, what I consider a limitation you might not. Since that's 
subjective it's probably best that we table further discussion on it 
since it comes down a matter preference.

>> Other than that I don't see why GtkD wouldn't work in D. Although on the
>> preceeding thread there were a bunch of people who were unhappy with it.
>> Specifically there were complaints about how well it worked on macOS and
>> Windows. I've used GTK apps on Windows before, and it was immediately
>> obvious to me that it was a GTK app, and it wasn't good. It functioned
>> but suffered from font sizing and box model problems. GTK is great if
>> you live in GNOME, outside of it, not so much.
> 
> There's an obvious solution there...
> 

... if I'm thinking what you're thinking, that is a pipe dream. I use 
GNOME too but sometimes we just have to accept that even if we're 
technically correct, it doesn't matter. :D

-- 
Adam Wilson
IRC: EllipticBit
import quiet.dlang.dev;