Input gathering: formalizing process of Lucee language evolvment

I moved 3 posts to a new topic: Trello vs Jira

As laid out in the OP, please refrain from discussing the merits of other peoples ideas while we gather them. There will be time for that later. The last four posts are a perfect example of why we should not do this (now nearly half of this thread is discussing Trello vs JIRA). Will try to sort this thread out in the morning - perhaps translpanting the Trello vs JIRA topic into its own thread; either way, it doesn’t belong here.

I fully agree with @dnando on this (and have expressed that with the appropriate likes).

I like to add to it though (not discuss the ideas) by saying the following:

The setup of the language advisory board has to be done by merit and competence and not by “Being an LAS member gets one a vote on the Language advisory board regardless”.

My justification for this is:

Being an LAS member expresses exactly one thing: that someone has an interest in Lucee that makes it a good commercial investment for them to pay the membership fees associated with it.

What it certainly does NOT express is that a member of LAS or their employees have any skill or qualification whatsoever to make language design decisions. There might be an overlap and that MIGHT well be the case, but paying a membership fee should NOT be the criteria to be involved with LuceeLang design work.

4 Likes

I moved 4 posts to an existing topic as I thought it was a good topic that could be focused in its own thread: Input gathering: who can help make language decisions, and how (e.g. Language Advisory Board)

In this thread, so far I think we have two key points that been raised (with some useful specifics):

  • An advisory board/group/committee, to help make decisions
  • Software to help provide visibility on the workflow of ideas raising through to implementation

Plenty of useful input, keep it coming :slight_smile:

New users in Discourse have (very) limited likes available, by the way…

I posted this in another thread but I am not sure it was noticed.

http://lang.lucee.org/t/the-missing-lucee-language-spec/126/28?u=apenhorwood

1 Like

@apenhorwood I took the liberty of moving your post into here. Seemed a good place for it. Is there anything in there that you think can be applied to process?

The point of the link was education on how others are handling the problems we’re facing. The whole site is a short read as the comments are clear and concise.

Here’s a rough idea of a flow for getting new input resolved through the wider community, a suggested advisory group and developers of the language (which could potentially be anybody).

This flow is my own invention and is intended as a visual for what’s in my head - it has not gone through LAS and does not represent them.

Detail could/should be added, such as linking issues with any advisory group’s recorded discussions to provide a trail (great point I think from @dnando The vote process itself needs detail. But anyways, here it is:

1 Like

That looks like a very good starting point imho. Do you have that file as kind of a source file for OmniGraffle or Visio or whatever you used to create it, @dom_watson ?

Yeah, gliffy - it won’t allow me to upload it here but here’s a pastebin of the raw file. Should be able to import it into Gliffy after saving as a .gliffy file: Lucee AG Workflow - Pastebin.com

Just a few questions:

  • In Jira, there is already the Proposal issue type. Is this workflow meant for that issue type, or are we talking about something else?
  • As far as the interaction between Jira and this forum, should proposal issues be spawned first, at which point discussion takes place on this forum, or should a discussion take place first, at which point a formal proposal issue is generated?

Another question,

Is there going to be a defined format for a proposal?

Using that type would definitely make sense. Those triaging issues would need to make the decision on what the issue type should be, the ‘Proposal’ type would be a good candidate for things that should go through the advisory group.

Good question, what I had in mind was that the JIRA issue would kick off that process. What leads to that JIRA issue being raised could be open discussion and someone being encouraged to raise the JIRA, or someone raising a JIRA on the behalf of someone else leading in from discussions here.

Similarly, if a JIRA proposal is raised without it first being discussed anywhere, I think it should be opened up for wider discussion before a decision is made.

There could be, though I personally think that guidelines may be more appropriate - do have any ideas around what format could be used?

I like the JEP (Java Enhancement Proposal) template. The only required sections on it are the summary and description, but it seems to be a good standard template overall.

1 Like

Proposals & Mini Specs

What to build, how it will be built, and managing resources as it is built are all discrete problems. This is a suggestion for how we might manage the “What to build” part.

Working on the assumption that the lifecycle of an idea is as follows:

  • someone has a brilliant idea; idea phase
  • its discussed ad nauseum; idea phase
  • the idea is run by the TAG to make sure its feasible; proposal phase
  • the idea is turned into a semi-formal spec for development; mini-spec phase
  • somebody builds the feature (QA, etc); mini-spec phase

Using lang.lucee.org

Keep discussions in the forum and not in JIRA.

Idea phase

Someone has a crazy idea. Put it up on the lang forum, and discuss.

Hooray! We’re doing this already.

Eventually the edges will be knocked off and the community will distill a decent summary from those for and against the concept. If the idea still has legs, it needs to enter a more formal process.

Note many ideas will just be that, and never proceed beyond an interesting discussion.

Proposal phase

An idea that has survived a robust discussion on the forum needs be turned into a Proposal.

The purpose of the Proposal is to distill a considered request from the community that the Technical Advisory Group (TAG) can comment on. A Proposal can cover any change in the server or underlying language.

The proposal should follow a basic template format (such as [JDK-8046186] JEP 2: JEP Template - Java Bug System), and embody a neutral statement of the feature with any pros and cons clearly articulated. It doesn’t need to be pulitzer prize winning; it just needs to be concise and unambiguous.

Proposals might only be created by users on the forum with a “trust level” of Regular, but anyone can reply to a proposal. Proposals would sit within their own sub-category, and would be editable WIKI posts. Discussion on proposals should be restricted to the clarification of the proposal only, not more belly-aching about the idea itself.

Replies that are more argumentative, can be moved to the original idea thread/topic. Replies that add to the Proposal may be moved into the WIKI post for clarity and hidden.

Proposals can then be formally reviewed by the TAG, and commented on.

Mini spec phase

Proposals that have passed community scrutiny and deemed viable by the TAG are “ready to build” and will become “mini specs”.

Calling them “mini specs” to set expectations about what’s suitable here; it should be anything that is a positive contribution to the server or language, even if its a relatively small change.

Specs get moved to their own sub-category by an admin or TAG rep. Further discussion on a spec should centre around coordinating efforts to build the agreed functionality.

Specs may or may not be picked up by LAS to build out of its limited development budget. Other development teams may want to coordinate with the core committers to submit a pull request for a new feature.

Having an agreed spec gives us all some clarity on what could be built. And the ability to better coordinate a wider group of committers. It also gives us an opportunity to better “sponsor” specific features, whether that is LAS or another party providing the resources.

1 Like

I like the @modius’s general outline, however it seems that this is entirely forum-centric? When does an issue in the bug tracker actually get raised?

Here’s my thoughts. The idea phase is dead on, however, once we enter the proposal phase, an issue needs to be generated. I personally think proposals/specs should have their own project in Jira. Oh, and no work should ever be done against a proposal issue.

So lets say a new project, LSPEC, is created. There are three issue types: Proposal, Mini-spec, and Major-spec. Proposal issues can be created by anyone, while *-spec’s can only be created by a yet-to-be defined group of trusted individuals (the TAG I assume?).

A possible workflow for these issues could be:

  1. Proposals are created with a “new” or “pending” status
  2. (Assuming this is possible) Proposals are locked for comments until they enter a “open for discussion” status.
  3. Proposals that do not meet requirements, or in some way are non-starters are rejected and closed by the project manager.
  4. If there are no major objections, the proposal’s status is changed to “open for discussion”.
  5. During the discussion phase, any appropriate edits to the proposal’s text can be made.
  6. Given an appropriate amount of time (say a minimum of 7 days) the issue is put into a “pending approval” status.
  7. It is at this point the TAG votes on the issue.
  8. There could be several possible outcomes, 3 that come to mind are: 1) Closed-Approved, 2) Closed-Rejected, and 3) Amended (requires further discussion)
  9. Once a proposal issue is approved, a mini-spec issue is created based on the proposal’s text, or referencing the proposal issue.
  10. Mini-spec issues are immediately closed.
  11. When it comes time to draft the next major specification, a new major specification issue is drafted, and selected mini-specs are assigned as children.
  12. The major-spec issue is then voted on by the TAG, and once approved, is given a version number.
  13. Back on the LDEV project, Feature issues are raised that reference any mini-spec’s that they support.

Using Jira this way, you now have traceability from proposal all the way through work completed. Also, language specifications are tracked separately from actual language development. You can then mark the server version as conforming to spec X, or partially conforming, etc…

1 Like

Just wondering if any one has seen PHP’s RFC process? I happened to bump into it and it appears as fairly clear, open, and robust process. I have not had any experience with it myself but I think it is worth a review/read.

https://wiki.php.net/rfc

An example of an approved language change https://wiki.php.net/rfc/isset_ternary

1 Like

I like the 3 phases suggested by @modius

  1. Idea Phase
    I think we are in agreement that this phase should happen on the forum: lang.lucee.org
  2. Proposal Phase
  3. Mini-Spec Phase

It is just steps 2 & 3 that we need to work on.

I’ve drawn this up into a flow diagram that I think represents your proposal @modius. Is this close to what you had in mind?

The Gliffy source for the diagram can be found here should anyone want to edit it and take it further (save as a .gliffy file):

2 Likes

That’s really informative. Here’s their page on “How to create an RFC”: PHP: rfc:howto.

I wonder if @dajester2013’s motion of using JIRA to track a proposal could be used to similar effect. i.e. you could have an RFC issue type with a specific workflow. Discussion happens in Lang, formalizing the spec happens in JIRA. A board could be created for RFC visibility.

I’m torn personally - there’s a real clarity in the PHP RFC pages that you might lose in JIRA (with all the UI around it). But the process may be easier to maintain and use in JIRA.