Compiler patch for runtime reflection

[Adam Wilson]

On Fri, 28 Oct 2011 00:07:57 -0700, Jacob Carlborg <doob at me.com> wrote:

> On 2011-10-27 20:42, Adam Wilson wrote:
>> On Thu, 27 Oct 2011 00:14:35 -0700, Jacob Carlborg <doob at me.com> wrote:
>>
>>> Are you saying that you consider using D for this Horizon project? I
>>> can recommend you take a look at DWT: www.dsource.org/projects/dwt
>>>
>>> Somewhere down the road I've planed to create an interface/window
>>> builder for DWT using XML or something similar. I'm thinking something
>>> like how it works on Mac OS X using Interface Builder.
>>
>> I am, and I have looked at DWT. My problem with it is a one that is
>> endemic to open-source UI framework. Microsoft recognized a decade ago
>> that UI widgets whose look-and-feel is defined and controlled by the
>> Operating System are going the way of the dodo. Out of that realization
>> WPF was born. UI designers today want the ability to control every pixel
>> of the UI's presentation. And the reasons for this are two-fold. The
>> first is that it turns out that most OS designers are fantastically bad
>> at UI and design in general. It takes epic piles of cash to pull of a
>> decent one and even then there is a still a "programmers were here" look
>> to it (with the notable exception of iOS/OSX where designers rule the
>> roost).
>
> I can agree to that.
>
>> The second is product differentiation. Nobody wants an app that
>> looks like every other app because it actually becomes impossible for
>> the user to distinguish which app works best for them.
>
> I want that. Because I know how the GUI works and it will be easy for me  
> to learn if it follows the guidelines of the platform. Also I don't need  
> to figure out if I can use the scroll wheel on the mouse on this, what  
> looks like a, scroll bar. If the application have a native look and feel  
> I know how to use the widgets.
>

Ahh, thats the true power of WPF. Presentation and Implementation are  
cleanly separated. I recent re-skinned every scrollbar in one of our WPF  
apps by writing the 30-line style once, placing it in the global resource  
dictionary with implicit styling enabled, and bam, every scrollbar in the  
app looks different, but functions in the exact same way any windows user  
would expect. There are cases where you can write code that calls up into  
the presentation layer and thus introducing a dependency on the  
presentation layer, but this is actively discouraged, and when I found one  
recently in a 3rd party vendors control, it was removed by the next  
release.
I can go on and on, but the point is, there are only two frameworks in  
existence that can do that are the WPF family and JavaFX. JavaFX is  
pathetic compared to WPF and is of course tied to an Oracle *shudder*  
controlled langauge. And a few months ago we had thousands of developers  
on the Silverlight forum looking for WPF family alternatives, all that  
they could find was JavaFX.

>> Users ONLY look
>> at the UI, and if the app doesn't look good, they wont "buy" it, even if
>> it's free. This is non-negotiable. Users, when given two apps that do
>> the same thing, even for different prices, will pick the prettier one
>> every time, because the prettier one is perceived as being "better".
>> It's called the Attractiveness Bias and it is a well-known principle in
>> the design world. Who would you rather look at all day, Alessandra
>> Ambrosio or Rosie O'Donnell? I rest my case.
>
> That's why I use Mac OS X where the native applications look good :)
>
>> I maintain that this is prime reason that Linux on the desktop has
>> failed miserably, and I think Android proves my point. Android's key win
>> is that it put a usable UI on top of Linux. People never had a problem
>> with the price of Linux, they just couldn't stand to look at it. The
>> Linux Desktop LOOKS industrial and it's apps for the most part look the
>> same (I know of a few outliers that did a good job, but it isn't the
>> norm).
>
> I can agree to that.
>
>> My point is that the day of cookie cutter apps is over. Anyone
>> designing for that paradigm is history. Microsoft's latest UI paradigm,
>> "Metro", is just a different cut-down version of WPF similar to
>> Silverlight. Microsoft has no plans to go back to OS controlled UI
>> styles; Metro and WPF are the plan for the next 15 years. (I attended
>> the MS BUILD conference, they made this plan very ... nay, EXTREMELY
>> clear).
>
> I don't know how Metro or WPF is implemented but how says they can't be  
> the native look and feel of the OS.
>

Nobody, in fact WPF has three default looks out of the box, one for  
Windows 2000, another for WinXP, and another for Aero (Vista+). WPF  
defaults to the correct one for that system unless you override it. It's  
just that in my experience, the default look is a lowest common  
demoninator, and not a good one.

>> Open-source is chronically behind the big boys with money, precisely
>> because FOSS doesn't have the money to sling around for Testing and
>> Usability Studies, and most FOSS guys don't want to mess around with
>> that stuff anyways. You see FOSS guys tend to be engineers; they can put
>> up with, and even like, industrial looking interfaces. But programmers
>> also have a giant blind-spot when it comes to users. Most programmers
>> view users as a lower species and assume that they will be delighted by
>> whatever the programmer deigns to bequeath to them. But if you look at
>> the successful people in the tech industry (*ahem* Steve Jobs) you'll
>> find an attitude that is the exact opposite. Jobs was so focused on
>> delivering what the user wanted that he would publicly berate any
>> programmer who thought they knew better than the designer. While I don't
>> necessarily agree with Jobs' management style, there is a reason why
>> Apple is the second largest company in the world right now, and it has
>> nothing to do with how good Apple's engineering is (which I hear is
>> average at best). Despite programmers' best efforts, the world of
>> technology is no longer controlled by programmers. For better or worse,
>> users determine our course now. The open-source community would do well
>> to embrace the user.
>
> I have no trouble in letting a designer decide how GUI should look like,  
> in fact, as you say it would be better if they did. But not everyone can  
> have that luxury. Since I'm no designer you do the best I can.
>

Neither am I really, I am a programmer by trade who cut his teeth back in  
the C++ days before C# came along and made me actually productive, but  
design isn't that hard, if one is willing to let go of their preferences.  
More to the point though, my company has no on staff designer, so it was  
either pay large sums of money to a consultant, or I as team lead could do  
it. I lost. To some degree it's a totally different way of thinking and I  
find that after a good design session (a week or two), it takes me a  
couple of days to realign my brain to left-brain-progammer-mode. But hey,  
they pay me, I jump to. *shrug* There is no mystical powers that are given  
to designers, anyone can make a good design.

>> But without a first-class UI framework, that will never happen. In terms
>> of capability and usability, both Apple and Microsoft have beat the best
>> FOSS has to offer by a decade at least. I looked, searched, and scoured,
>> but the fact of the matter is, even the usable FOSS UI offerings are
>> pitiful when compared to the commercial offerings. The Horizon Project
>> got it's start because there has been a trend in recent commercial UI
>> offerings from towards increasing reliance on the operating system
>> itself. Metro XAML just flat won't work on anything other than Windows
>> 8, Silverlight is a second class citizen at MS now, Cocoa only works on
>> Mac etc. My goal with The Horizon Project is to create First-Class UI
>> Framework for multiple platforms so that programmers don't have to
>> rewrite the UI from scratch for each new platform they want to support
>> and then open-source it so that the commercial OS vendors can't pervert
>> it for their own purposes. I want to put [some of] the power back into
>> the programmers hands.
>>
>> Apologies for the length, but this is a topic that is of some interest
>> to me. :-)
>
> I's a topic of interest to me as well. But I, on the other hand, prefer  
> a native look and fell of applications. Instead of having yet another  
> application with yet another GUI that doesn't work properly. If I see  
> something that looks like a scroll bar I assume I can use it like a  
> scroll bar. But that's not true for many applications, instead they  
> implement the bare minimum for having the scroll bar "work", i.e. I may  
> not be able to scroll using the wheel on the mouse. A great example of  
> this are scroll bars in games.
>
> In my experience of non-native GUI's they always perform worse the  
> native GUI's.
>

Actually we found that by re-skinning our app intelligently we were able  
to load a large list of complex data 4x faster than the default skin. The  
reason for this is that in WPF all drawing is done the same way, it  
doesn't matter if it's a system skin or not, it's just a skin, there are  
no special shortcuts for the system skin to take. At it's heart WPF is  
based on DirectX9. This means that screen complexity is the ONLY limiting  
factor.

The other thing that WPF allows is data / presentation separation. I can  
pass a list box a chunk of data, and the data does not have to concern  
itself with how the presentation looks or vice versa. The listbox will  
display the data according to however the style is setup it. And THEN if I  
really want to I can change the presentation style on the fly based on  
user input. Can DWT do that in 2 lines of code? WPF Can. That's the power  
of reflection based data-binding. In fact I spend most of my time writing  
(much more compact) XAML code to do presentation layer work than I do  
write C#, to the point where the C# coding makes up less than half of the  
work done on a screen. It's usually things like load this item based on a  
selection changed event or some such that XAML has no way to do (and  
shouldn't, thats implementation).

> Anyway, you can still use DWT to draw your own widgets. Even if you  
> don't use any native widgets you still need to be able to draw something  
> on a surface. DWT has a platform independent way of doing that.
>

Yea, but how much work is it? Can I create a completely new look for a  
control in 30 lines or less without modifying the implementation code at  
all to make the new look work? That's the power I'm after.

> IBM Lotus Software and IBM Rational Software are built on SWT (which DWT  
> is a port of) uses a lot a non-native widgets. There's an application  
> called "Azureus" that had its own GUI that didn't look native at all,  
> apparently it's called Vuze these days and it looks native.
>

That's IBM, they have supertanker loads of money, the open-source world  
does not. I expect IBM to re-skin anything they want to their  
specifications, they can afford it. WPF type apps are game changers in  
that they allow very small teams to create powerful, beautiful, apps in  
something less than a decade, and without requiring multi-million dollar  
investments. As an example, our flagship app (which we are keeping under  
tight wraps for the following reason) has the potential to completely wipe  
out multiple billions of dollars worth of corporate market capitalization  
and probably sink some long-time industry names in the process, all with a  
team  of less than 10 guys. At least, thats our goal. But that goal is  
only achievable because we can move orders of magnitude faster than they  
can. It also doesn't hurt that their interface looks like it was designed  
in 1999 (seriously) and is definitely a traditional type of framework (the  
standard Java one IIRC). Thank you WPF, it simply wouldn't be possible  
without you. So when I say the world is leaving behind traditional UI  
frameworks, I am not kidding; it just wont happen that quickly as corp IT  
loathes change, but it's already starting. (See: iOS and Android apps and  
CxO's demanding their internal apps on those platforms.)

Now imagine a world where not only can open-source compete with the  
big-boys, but take serious chunks out of their consumer mind-share. I want  
that. But, the only way that is happening is with a framework that allows  
open-source to compete at the same speed as the big boys can afford while  
delivering what the customer wants. Any app is only as good as the number  
of people using it.

It's not the big who eat the small, it's the fast who eat the slow. :-)

>>> I don't understand this one. Should the compiler disable reflection as
>>> soon as it sees malloc/free? On what level should it be disabled? I
>>> mean, the runtime needs to be able to use these functions to implement
>>> the memory manager, i.e. the garbage collector and other things as  
>>> well.
>>
>> The idea is to automatically prevent reflection access to sections of
>> the program where using direct memory manipulation could potentially
>> result in security holes but provide a way out for the programmer if
>> they really wanted it. C/C++ is famous for programmer bugs around memory
>> handling. If the compiler automatically disabled reflection, by sticking
>> @noreflect in front of a function that used malloc or new, it could
>> potentially prevent those types of memory manipulation flaws and help
>> keep the Reflection attacks to a minimum. It was an idea that I was
>> throwing out there. But I don't know enough about D yet to know if it's
>> the right way to handle it. And I have to admit I am little confused
>> though as I would hope that Reflection would be disabled on the GC,
>> because I have never personally had a reason to reflect into the GC...
>
> What happens when you use class A from class B and the compiler has  
> added @noreflect to class A? Will it add it to B as well? If not, how  
> does the compiler know that?
>
> There has been a similar discussion about having the compiler insert  
> "pure" automatic on functions. But what might happens is a function that  
> is called by another function change it implementation making it no  
> longer pure. Which means your function will no longer be pure and you  
> have no idea about it.
>


-- 
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.com/