Strategies for resolving cyclic dependencies in static ctors

[Max Samukha]

On 03/22/2011 02:12 AM, Nick Sabalausky wrote:
> I'm intending this thread as somewhat of a roundtable-like discussion.
> Hopefully we can come up with good material for a short article on Wiki4D,
> or maybe the D website, or wherever.
>
> The scenario: A coder is writing some D, compiles, runs and gets a "Cyclic
> dependency in static ctors" error. Crap! A pain for experienced D users, and
> very bad PR for new D users. (Hopefully we'll eventually get some sort of
> solution for this, but in the meantime D users just have to deal with it.)
>
> The question: What now? What strategies do people find useful for dealing
> with this? Any specific "first steps" to take? Best practices? Etc.

One commonly used hack is to move static constructors into a separate 
helper module and call the initialization function via a C extern (like 
it is done in std.stdiobase):

----
module foo_helper;

private extern(C) foo_static_ctor();
static this()
{
     foo_static_ctor();
}

-----

module foo;
import foo_helper;

private Object global;
private extern(C) void foo_static_ctor()
{
     global = new Object;
}
----

Note that "global" is guaranteed to have been initialized when accessed 
from static constructors in modules that import "a". Being able to 
instruct the compiler to do this implicitly (so we could put static 
ctors in templates, for example) would probably solve most of static 
ctor problems.

>
> Aside from the old "start merging modules" PITA that many of us are familiar
> with from the old "100 forward reference errors at the drop of a hat" days,
> I've found one viable (but still kinda PITA) strategy so far:
>
> 1. Look at the line that says: "module foo ->  module bar ->  module foo"
>
> 2. Pick one of those two modules (whichever has the simplest static ctors)
> and eliminate all the static ctors using the following pattern:
>
> The pattern: The trick is to convert every variable that needs to be
> initialized into an "init on first use" ref @property. This "ref @property"
> checks a "hasThisBeenInited" bool and, if false, runs all the
> initialization. If the variable is a reference type, then *sometimes* you
> can get away with just checking for null (though often not, because it will
> just get-reinited ).
>
> Example:
> ------------------------------
> // Old:
> class Foo { /+...+/ }
> int number;
> Foo foo;
> static this
> {
>      foo = new Foo();
>      number = /+ something maybe involving foo +/;
> }
>
> // New:
> class Foo { /+...+/ }
>
> private int _number;
> @property ref int number()
> {
>      forceModuleInit();
>      return _number;
> }
>
> private Foo _foo;
> @property ref Foo foo()
> {
>      forceModuleInit();
>      return _foo;
> }
>
> bool isModuleInited=false;
> static void forceModuleInit() // Hopefully inlined
> {
>      if(!isModuleInited)
>      {
>          staticThis();
>          isModuleInited = true;
>      }
> }
> static void staticThis() // Workaround for static ctor cycle
> {
>      foo = new Foo();
>      number = /+ something maybe involving foo +/;
> }
> ------------------------------
>
> If one of the variables being inited in the static ctor is something you
> *know* will never be accessed before some other specific variable is
> accessed, then you can skip converting it to "@property ref" if you want.
>
> It's a big mess, but the conversion can be done deterministically, and even
> mechanically (heck, a ctfe string mixin wrapper could probably be built to
> do it).
>
> The potential downsides:
>
> 1. If you come across an @propery bug or limitation, you're SOL. This should
> become less and less of an issue with time, though.
>
> 2. If one of the variables you converted is frequently-accessed, it could
> cause a performance problem.
>
> 3. Small increase to storage requirements. Might potentially be a problem if
> it's within templated or mixed-in code that gets instantiated many times.

4. Initializing shared data needs synchronization. Then, your example 
would look similar to this:

------------------------------

class Foo { /+...+/ }

private immutable int _number;
@property ref int number()
{
      forceModuleInit();
      return _number;
}

private immutable Foo _foo;
@property immutable(Foo) foo()
{
      forceModuleInit();
      return _foo;
}

bool isModuleInited=false;
__gshared bool isSharedModuleInited=false;
static void forceModuleInit() // Hopefully inlined
{
      if(!isModuleInited)
      {
          synchronized(someLock)
          {
              if (!isSharedModuleInited)
              {
                  staticThis();
                  isSharedModuleInited = true;
              }
          }
          isModuleInited = true;
      }
}
static void sharedStaticThis() // Workaround for shared static ctor cycle
{
      auto foo = new Foo();
      _number = /+ something maybe involving foo +/;
      _foo = cast(immutable)foo;
}
------------------------------

>
> At one point, I fiddled around with the idea of converting static ctors to
> "staticThis()" and then having one real static ctor for the entire library
> (assuming it's a library) that manually calls all the staticThis functions.
> One problem with this is that it's easy to accidentally forget to call one
> of the staticThis functions. The other big problem I found with this though,
> especially for a library, is that it requires everyone importing your code
> to always import through a single "import foo.all" module. If some user
> skips that, then the static ctors won't get run. There might be some
> possible workarounds for that, though:
>
> - If the library has some primary interface that always gets used, then that
> can easily check if the static ctors have run and error out if not. If the
> primary interface is *always* the first part of your library used (or at
> least the first of all the parts that actually rely on the static ctors
> having run), then you could even run the static ctors right then instead of
> erroring out. That's a lot of "if"s, though, so it may not be
> widely-applicable.
>
> - If you convert *all* static ctors to staticThis(), it might be possible to
> stick the one main static ctor into a private utility module that gets
> privately imported by all modules in the library. Then users can continue
> importing whatever module(s) they want. But if you don't convert *all* of
> the static ctors to staticThis, then you'll just re-introduce a cycle.
>
> But if there's ever two separate libraries that have any interdependencies,
> then the one-main-real static ctor (that calls all the staticThis() funcs)
> will have to be shared between the two libraries. So overall, this approach
> may be possible, but maybe only in certain cases, and can involve a lot of
> changes.
>
>
>