Input gathering: formalizing process of Lucee language evolvment

Please use this thread to voice ideas around how LAS can formalize the process of language development. This input can be taken from working examples from other languages, or indeed, anything that comes to your mind. All input is welcome within the following “rules”:

Rules

  1. Please only provide:
    i. Ideas
    ii. further development on existing ideas
    iii. requests for more information from participants (i.e. where you feel that they may have more to give or you don’t understand what is being said)
  2. Please do not:
    i. Provide a critique of ideas (it is important that ideas be critiqued later, but at this point they should be allowed to breathe)
    ii. Repeat ideas that have already been made
  3. If you like an idea, click on the heart button rather than replying with “+1” or “I agree”, etc. We can use that to roughly guage how we all feel about ideas.

The purpose of these rules is to allow the community to air their thoughts in a focused way. Once we feel that the thread is winding down, we will gather the ideas and see how we can progress them (at which point a critique of the ideas can happen).

I will monitor this thread and enforce the rules if necessary (though I’m sure it won’t be).

1 Like

If we all worked in the same office, and we were all somewhat competent at language design, we’d very likely find a way to develop an agenda of points to discuss, so the end product was a list, and then schedule regular meetings, discuss the pros and cons face to face, and reach a decision.

If we tried to do that over a forum or via an email list, it would take a lot longer and be more likely to lead to polarizing disagreements. Typing away, all we each have are imaginary parties that may be disagreeing with our point in imaginary ways. We all can get lost in our own echo chambers that way - it’s almost inevitable.

I don’t know how other communities manage this, but here’s a rough proposal:

  1. Collect a list of language issues and ideas via a simple online application. There should be some constraints on this process so it doesn’t get out of hand, but generally anyone should be able to contribute.
  2. Create a language advisory board made up of qualified people.
  3. Advisory board meets online using Google Hangouts or another similar service that can best simulate a face to face meeting over the internet, discusses each point on the language issues list and reaches a (provisional) decision on issues (or postpones the decision, perhaps inviting community feedback or recommending further research). The meeting is recorded.
  4. The recorded meeting is then posted online which would provide full transparency and allow feedback from the community at large. Such feedback would only be allowed after the advisory board discusses an issue, and those wanting to give feedback would be required to listen to the discussion beforehand so they are fully informed. There would need to be a time limit imposed on community feedback so that final decisions can be made.
  5. At the next meeting of the advisory board, community feedback is reviewed, discussed, and a final decision is made (or delayed if need be) on issues where feedback is given. Issues where no feedback has been received would be noted as decided on a final basis if appropriate. New language issues are discussed.
  6. The advisory board will need some internal structure to resolve disagreements that all members agree to as a condition of their participation. Other communities refer to this as a “benevolent dictator”. I’m not suggesting that the final decision maker be a single person necessary, only that a structure is decided upon when the advisory board is formed to resolve issues that members are split on. When a consensus can’t be reached on a particular issue, those on the advisory board will need to agree to accept the final decision gracefully.

Primary objectives behind this idea are efficiency and clarity of communication, transparency, and a structured approach to community involvement.

4 Likes

One thing I’ve been thinking that might help is using a JIRA workflow with issue types that can be used to provide visibility into what is being discussed and what phase of discussion items are at. A kanban board could be created to give clear visibility.

This might work for channeling items of discussion into (and out of) an Advisory Board for instance.

1 Like

A small refinement. The language issue app would of course need to record which meeting, or meetings, each issue was discussed at, via a url to the Google Hangout, so we all knew where to go to listen to a particular discussion. This bit of information would also provide an audit trail so that in future we all can revisit how a particular decision was made. That way, if I come along and say “Why was RecordCount removed from query!!! That was such a stupid decision!”, you all can simply point me to the issue tracker and say "Go listen to the discussion. There were good reasons … " This might help prevent us from rehashing the same points again and again.

3 Likes

One way to do that might be to use Trello. I see Ghost use trello for all their feature gathering and roadmap. Check it out here:

Edit (posted too early): This way the language can evolve. Not just initial release but over time, you can see WHY people reached to a certain decision (in the Released column) and you can see a backlog of ideas that got there.

If you click on the first item in the backlog you get more information (copying here for the link weary) on how to use it:

Description
This is a work-in-progress roadmap of all the things we are currently adding (and plan to add) to the Ghost publishing platform. As we start planning what to work on, features move from left to right until they’re completed and ready for everyone!

Voting for Features

If you have an idea that you want to see on our roadmap - head over to http://ideas.ghost.org and create it or vote for it!

Columns

Backlog = big unsorted “ToDo” bucket
Next = things we’re going to work on next
In progress = things we’re working on right now
Recently released = things we have already released
Labels

Label colours indicate the current status of a card in our roadmap.

Green = Currently being discussed / spec’d
Yellow = Currently being designed
Orange = Currently being developed
Red = Stalled/On-hold
Purple = Currently being reviewed
Blue = Ready/done
Feedback

If you’re stuck, check out our user docs and developer docs where you’ll find tons of helpful info.

Self hosted users can get support via our support forums - while Ghost(Pro) users can always email us directly on support@ghost.org

If you just want to tell us how awesome Ghost is (please do, it makes us smile), drop us a line on Twitter or Facebook or Google+

1 Like

I moved 3 posts to a new topic: Trello vs Jira

As laid out in the OP, please refrain from discussing the merits of other peoples ideas while we gather them. There will be time for that later. The last four posts are a perfect example of why we should not do this (now nearly half of this thread is discussing Trello vs JIRA). Will try to sort this thread out in the morning - perhaps translpanting the Trello vs JIRA topic into its own thread; either way, it doesn’t belong here.

I fully agree with @dnando on this (and have expressed that with the appropriate likes).

I like to add to it though (not discuss the ideas) by saying the following:

The setup of the language advisory board has to be done by merit and competence and not by “Being an LAS member gets one a vote on the Language advisory board regardless”.

My justification for this is:

Being an LAS member expresses exactly one thing: that someone has an interest in Lucee that makes it a good commercial investment for them to pay the membership fees associated with it.

What it certainly does NOT express is that a member of LAS or their employees have any skill or qualification whatsoever to make language design decisions. There might be an overlap and that MIGHT well be the case, but paying a membership fee should NOT be the criteria to be involved with LuceeLang design work.

4 Likes

I moved 4 posts to an existing topic as I thought it was a good topic that could be focused in its own thread: Input gathering: who can help make language decisions, and how (e.g. Language Advisory Board)

In this thread, so far I think we have two key points that been raised (with some useful specifics):

  • An advisory board/group/committee, to help make decisions
  • Software to help provide visibility on the workflow of ideas raising through to implementation

Plenty of useful input, keep it coming :slight_smile:

New users in Discourse have (very) limited likes available, by the way…

I posted this in another thread but I am not sure it was noticed.

http://lang.lucee.org/t/the-missing-lucee-language-spec/126/28?u=apenhorwood

1 Like

@apenhorwood I took the liberty of moving your post into here. Seemed a good place for it. Is there anything in there that you think can be applied to process?

The point of the link was education on how others are handling the problems we’re facing. The whole site is a short read as the comments are clear and concise.

Here’s a rough idea of a flow for getting new input resolved through the wider community, a suggested advisory group and developers of the language (which could potentially be anybody).

This flow is my own invention and is intended as a visual for what’s in my head - it has not gone through LAS and does not represent them.

Detail could/should be added, such as linking issues with any advisory group’s recorded discussions to provide a trail (great point I think from @dnando The vote process itself needs detail. But anyways, here it is:

1 Like

That looks like a very good starting point imho. Do you have that file as kind of a source file for OmniGraffle or Visio or whatever you used to create it, @dom_watson ?

Yeah, gliffy - it won’t allow me to upload it here but here’s a pastebin of the raw file. Should be able to import it into Gliffy after saving as a .gliffy file: Lucee AG Workflow - Pastebin.com

Just a few questions:

  • In Jira, there is already the Proposal issue type. Is this workflow meant for that issue type, or are we talking about something else?
  • As far as the interaction between Jira and this forum, should proposal issues be spawned first, at which point discussion takes place on this forum, or should a discussion take place first, at which point a formal proposal issue is generated?

Another question,

Is there going to be a defined format for a proposal?

Using that type would definitely make sense. Those triaging issues would need to make the decision on what the issue type should be, the ‘Proposal’ type would be a good candidate for things that should go through the advisory group.

Good question, what I had in mind was that the JIRA issue would kick off that process. What leads to that JIRA issue being raised could be open discussion and someone being encouraged to raise the JIRA, or someone raising a JIRA on the behalf of someone else leading in from discussions here.

Similarly, if a JIRA proposal is raised without it first being discussed anywhere, I think it should be opened up for wider discussion before a decision is made.

There could be, though I personally think that guidelines may be more appropriate - do have any ideas around what format could be used?

I like the JEP (Java Enhancement Proposal) template. The only required sections on it are the summary and description, but it seems to be a good standard template overall.

1 Like

Proposals & Mini Specs

What to build, how it will be built, and managing resources as it is built are all discrete problems. This is a suggestion for how we might manage the “What to build” part.

Working on the assumption that the lifecycle of an idea is as follows:

  • someone has a brilliant idea; idea phase
  • its discussed ad nauseum; idea phase
  • the idea is run by the TAG to make sure its feasible; proposal phase
  • the idea is turned into a semi-formal spec for development; mini-spec phase
  • somebody builds the feature (QA, etc); mini-spec phase

Using lang.lucee.org

Keep discussions in the forum and not in JIRA.

Idea phase

Someone has a crazy idea. Put it up on the lang forum, and discuss.

Hooray! We’re doing this already.

Eventually the edges will be knocked off and the community will distill a decent summary from those for and against the concept. If the idea still has legs, it needs to enter a more formal process.

Note many ideas will just be that, and never proceed beyond an interesting discussion.

Proposal phase

An idea that has survived a robust discussion on the forum needs be turned into a Proposal.

The purpose of the Proposal is to distill a considered request from the community that the Technical Advisory Group (TAG) can comment on. A Proposal can cover any change in the server or underlying language.

The proposal should follow a basic template format (such as [JDK-8046186] JEP 2: JEP Template - Java Bug System), and embody a neutral statement of the feature with any pros and cons clearly articulated. It doesn’t need to be pulitzer prize winning; it just needs to be concise and unambiguous.

Proposals might only be created by users on the forum with a “trust level” of Regular, but anyone can reply to a proposal. Proposals would sit within their own sub-category, and would be editable WIKI posts. Discussion on proposals should be restricted to the clarification of the proposal only, not more belly-aching about the idea itself.

Replies that are more argumentative, can be moved to the original idea thread/topic. Replies that add to the Proposal may be moved into the WIKI post for clarity and hidden.

Proposals can then be formally reviewed by the TAG, and commented on.

Mini spec phase

Proposals that have passed community scrutiny and deemed viable by the TAG are “ready to build” and will become “mini specs”.

Calling them “mini specs” to set expectations about what’s suitable here; it should be anything that is a positive contribution to the server or language, even if its a relatively small change.

Specs get moved to their own sub-category by an admin or TAG rep. Further discussion on a spec should centre around coordinating efforts to build the agreed functionality.

Specs may or may not be picked up by LAS to build out of its limited development budget. Other development teams may want to coordinate with the core committers to submit a pull request for a new feature.

Having an agreed spec gives us all some clarity on what could be built. And the ability to better coordinate a wider group of committers. It also gives us an opportunity to better “sponsor” specific features, whether that is LAS or another party providing the resources.

1 Like