p4

https://futureofcoding.org/log#prototype-4

The future work section of my paper talks about visual metaphors for FRP. While I do think this is quite important in order to "democratize visual intuition" (Penrose), I wonder if it's necessary. What if we take normal FRP haskell-ish notation as a starting place and simply augment it with LIVE-ness, such as an interpreted environment, showing data, and evaluating as far as possible even when there are holes (Cyrus Omar's work). Geoff Litt and Paul Chiusano suggested that expressions with holes are just functions with those holes as parameters, and we could put a slider or examples values in there. Always concretions, never pure abstractions.

Here's another idea I've been toying with: Jason's Brennan's notion of a programming environment on an endless canvas, like in Sketch or Photoshop. One thing this could enable is a structure-less structured editor - in that you could put together pieces of expressions in various places and combine them later. I guess this would be similar to Scratch or Blockly... which I don't love... One possibility is to use the layer metaphor from Photoshop as a programmatic abstraction, but I don't know what that would mean exactly yet.

My first thought was to build this as a FP thing first and build my way up to FRP. It could be the data slice-and-dice ninja thing, starting with JSON/CSV slice, dicing, and joining -- this is related to datafun.

But then I went ahead and did a drawings for it as an FRP platform:

The goal is to submit this work to <Programming> in Feb and a good title for the PX workshop (says JE) is "FRP eXperience".

There are a few different levels here:

the abstraction / code
the streams, including data and how they fit together
the output UI

Infinite canvas drawings

Other tools for inspiration

WYSIWYG hybrids such as Sketch-n-Sketch
Aprt.us, which would be a more visual language. Key question is how to do higher-order & cyclic, and other abstractions
Facebook Origami
Lamdu/Luna/Dark and all the projectional editors

Worst parts of Reflex/Haskell

But then I thought: let's focus on the really key issue here. What's the highest-leverage improvement to be made? What's the worst part of using Reflex/Haskell now? In the shower note on the right below, it says:

feedback loop: knowing if the syntax is right, if the types make sense, what the code I'm looking at is from both a type, data, and UI perspective
knowing what's possible (syntax, API, types)

p4 prioritized features

With this framing, a quick compile time (or intepreter time) could be key. Reflex seems to take 4+ seconds. Can I do sub 1? Or faster? How about hot reloading?
Great docs, autocomplete, type annotations everywhere
Automatic plumbing: can we semi-automatically lift transforms inside monadic contexts? Ditto for automatic type coersion plumbing like int to string, etc
Stream visualization, including how streams make other streams, the shape of streams, and also the data for each stream
(Bi-directional) Direct manipulation of UI

Implementation

JE send me an email:

Can you implement your FRP eXperience using FRP, and apply it to itself? That might be challenging, but would also be very impressive. It will be seen as a limitation if it can't apply to itself, although that might be unavoidable in the first phase. But I think that at the very least you need to implement it in some FRP framework to avoid the charge of hypocrisy.

My response: The only implementation of the FRP I argue for in my paper is Reflex/ghcjs. It doesn't exist in PureScript, Elm, F#, etc. Turns out laziness makes it a lot easier to implement. My next experiment is to try to do it in JS with CycleJS. I've tried this in the past. It's not easy -- it's not built for this use case -- but it seems possible, even cycles. More to follow...

I did a very naive implementation of streams stuff in JS here: https://codesandbox.io/s/81225lrop2
It'd probably be better to build off cyclejs. Over a year ago I tried to do something similar so maybe I can start there? https://codesandbox.io/s/xlrynkqoqp Inspecting the types of streams is key. Maybe I can use the cyclejs devtools and/or augment them somehow. Could typescript or something similar help me encode the types of streams of streams of streams? Could monaco help with autocomplete suggestions?

Some other ideas for building this:

racket turnstule dsl for building typed langauges
L. Kats and E. Visser. The spoofax language workbench.
C. Omar. Reasonably Programmable Syntax. PhD thesis, Carnegie Mellon University, 2017.
Build a JS interpreter with laziness based on Stephen Diehl's work (discussed below as well)

## Mid Nov 2018 Many Thoughts

making frp easier by having sequential semantics and we abstract to frp - like how joyJS can go from derivative iterative to continuous. logo and Nicky cases language and derivative thinking from inside vs implicit matjmstcal thinking from outside. two sides of same coin?

@stevekrouse

https://futureofcoding.org/log#p4-thinking-for-je-meeting

p4 thinking for JE meeting

TOC
{: toc }

Theme of my work

A fun question on Twitter yesterday encouraged me to think a bit broader as to an underlying theme to my work that would encompass this project, as well as other outside of "improving programming" (namely, LogicHub). I'm proud of what I came up with:

Offloading mental tasks better done by computers to computers, so humans are freed up to think creative thoughts
— Steven Krouse (@stevekrouse) November 21, 2018

Some responses about fluid Haskell

Sean McDirmid suggested Haskell for Mac, which is very cool and close to what I want so I installed it, but I wasn't able to see an easy way to get Reflex to work with it...

Luke Iannini apparently worked on a live recompiler for Haskell.

The monadfix people shocked me when they said:

None of us have been able to achieve the "fluid, live Haskell programming experience" that you're yearning for, even without "has to work GHCJS" as an additional constraint.

My new thesis is that this fluid Haskell doesn't really exist - at least for most Haskell developers.

Artyom from monadfix was also very kind and commented on my issue about installing intero. Too bad they can't help here!

Biggest experience problems with Haskell/Reflex

I made a full day's effort of trying to steelman Haskell/Reflex and make the experience as good as possible before trying to improve it - yet I just have such distaste for installing and debugging shit in the terminal (as well as using the existing Reflex setup I have) that I wonder if I can simply pull on my memory for the key issues...

1. Speed of feedback loop

The main thing was that speed of feedback on all fronts (syntax, types, output) was so slow and required so many keystrokes. In particular there were times that I could point to places in my code where I just wanted to know the type of something but did not know how to ask Haskell for that information.

2. API Discoverability

In other words, "I have some things. I want some other things. What blocks can I use to go from what I have to what I want?" In Scratch, all the "legos are on the floor" to help with this. I find that the lodash JS library also does a superb job of this. I found the Reflex documentation and the Haskell autocomplete tooling to feel like I'm basically guess and checking.

3. Plumbing code

Such as converting Int or String to Text and back, or worst of all, collapsing higher-order streams. I was very excited to find this extremely helpful video, Real World Reflex last night which has a really wonderful slide to help with this:

4. Visualize streams

This includes seeing the "shape" of streams and how streams make up other streams as in rxmarbles.com, as well as watching the live data flow through streams.

5. Direct manipulation

This whole thing won't feel natural without direct or bi-directional manipulation of the output because why not?

Next step ideas

1. Stay in Haskell

Continue working towards a fluid experience in Haskell/Reflex.

Find someone to help me setup a better experience and plow through.
Make better documentation for Reflex
Make better utility functions on top of Reflex so I don't have to do as much type conversion/plumbing
Make a devtool for Haskell/Reflex to visualize the streams
[I'd probably never get here] Use some of the bi-directional sketch-n-sketch work

2. Build/use JS library/framework

Investigate CycleJS
Look for another stream framework with higher-level and cyclic streams.
See if I can only allow consts and no object modifications while still getting cycles (maybe use a fix-like thing?)

3. Build/use JS Haskell interpreter

Investigate the two I already found here and here, the first of which comes with an online REPL
Experiment with them to see if I can achieve a fast feedback loop
Build a Reflex-like library on top of them

4. Build a higher-level system that compiles to JS

Sketch out what it would look like, pulling inspiration from Facebook Origami, the rx visualization tools, the animation tools such as Principle For Mac, Josh Horowitz's Pane and what it references including Aprt.us.
Prototype what "format" it would compile to in JS, be it a CycleJS data structure or something else
Build it...

I'm going to spend the next ~30 min doing the sketching discussed above in preparation for my meeting in a few hours with JE.

https://futureofcoding.org/notes/jonathan-edwards/11-21-18

they have generators in JS so maybe we have laziness and also weak refs maybe
- TODO ask ryan trinkle
building a langauge is a lot but a JS framework is great
- people would immediately download and play with it
interpreting haskell seems like a lot of work
starting from the top (the UI like aprt.us) is good
could start from a DSL for the streams --> maybe a macro thing for JS or a little lang with syntax and a parser with embedded JS
he's skeptical of the Xerox parc advice to build one's own infrastructure to build what you want on top
imagining UI is a pain but implementing is weeks or months of work, but a DSL can give you great sense of the UI and so much easier to build
- the risks with this is that you'll go with things that are good on paper but hard to visualize
- tons of DSL tools: spoofax, MPS, look into this for JS
strongly recommends typescript
- there's a lot of work on "functional typescript" and some even have typeclasses!
subtext in the past has been "parametric in my mind", living in multiple places in the design space

@conal

https://futureofcoding.org/log#fluidity-vs-structure

Fluidity vs structure

I need to spend some time defining fluidity and structure, but this graph feels provocative:

I think fluidity has to do with live-ness, feedback loop speed, incremental actions causing incremental result, etc. And structure has to do with possible / impossible states, types, schemas. I'm stuck on where AST editors fit in this layout: they enable greater feedback loop speed knowledge of errors and prevent error states, yet they are not as fluid as using text but text doesn't give you as much info as fast.

One interesting note is that Airtable is particularly noteworth as having high structure and also high fluidity.

Conal -> Turbine

Just as I decided (in the last meeting with JE) to build or find a JS lib for DCTP, I popped onto Twitter to find this:

Stumbled across another *spot-on* post about FRP by @paldepind: "Let's reinvent FRP" https://t.co/Ga7ywHnySI .
— Conal Elliott (@conal) November 23, 2018

Which led me back to Turbine, which I had found a few months back and mistakenly disregarded as the "wrong kind of FRP" as mistakenly interpreted their Now monad thing. I'm now quite excited about this library! I already sent two long emails to the creator, Simon, and hope he responds soon. Here's what I wrote I'd like to collaborate on:

Documentation. For example, I had to figure out how list worked from reading the source and puzzling together a few examples without explanations. I'd love to help document every function in the API. (Additionally, I believe the code itself could use some comment documentation but that's more up to you.)

Understanding the types of the streams I'm working with would help a lot. Maybe getting TypeScript set up (as I failed to do in the issue above) would help here.

Being able to "inspect" streams better as a debugging and understanding tool. CycleJS has this wonderful devtool and there are a number of other really cool stream visualization tools we can draw on for inspiration. At the very least, a better console.log story would go a long way. It was really tough to figure out what was going on with my streams.

Collapsing higher-order streams is really hard, but luckily this picture makes it a LOT easier. It saved my life last night as I was working on my favorite FRP problem of buttons that add buttons that add buttons... but only the odd buttons. Maybe we can build on this picture somehow or at least incorporate it into the documentation...

As stated in the issue above, I don't like the way the model and view are separated. I wonder if it's possible to combine them like in Reflex and other "original FRP" frameworks.

And after we/I work on these more pressing issues, the next step will be building a layer on top of that Turbine that would make the development experience much better. That is, building a GUI that "compiles" to Turbine, for example, kind of in the spirit of Conal's Tangible Functional Programming. Here I am referring to more radical ideas than improving the documentation or a simple devtool, such a projectional editor in the spirit of Lamdu, Luna, Isomorf, Dark, Unison, and Hazel.

https://futureofcoding.org/log#p4-thoughts-12318

p4 thoughts 12/3/18

Start with non UI motivating problems

A way to simplify this problem is to build an FP playground to solve normal FP problems with cyclical streams first and work our way up to UI:

Such as the problems that Pane solves
Here's an idea for a developer tool extension slice-and-dice playground thing. Here's a common pattern I found useful: querySelect some nodes and map/filter over them a bunch, such as:

[].slice.call(editorElement.querySelectorAll('*'))

  .map(e  =>  [e,  e.innerText])

  .map(([e,  text])  =>  [e,  text.match(/^(#+)\s.*$/)])

  .filter(([e,  m])  =>  m)

  .map(([e,  m])  =>  [e,  m[1]])

  .forEach(([e,m])  =>  {

    e.style.fontSize  =  (50  -  (5*m.length))  ||  3;

    e.style.marginBottom  =  2  +  "px";

    e.style.marginTop  =  2  +  "px";
  })

The issue with this approach is that we may create a FP playground that won't scale up to cyclical UI problems...

Fluidity is not the initial focus

I realize that part of why structured editors haven't been able to compete with text-based coding is:

Computers come with a hardware input device especially designed for text input: the keyboard!
We all have spent dozens of hours learning to use this keyboard for text input!

It's simply not a fair comparison to expect a new interface to be as fluid as text from Day 1. Of course fluidity is important and of course making it work with people's existing hardware and skills will help with adoption, but they aren't the first things to worry about. Maybe the ultimate interface will require new input hardware and/or a lot of practice to get the hang of. Hopefully we can get away with the keyboard and mouse and make the onboarding simple, but we don't want to pigeonhole ourselves over it.

The initial focus should be on the comprehensibility of the code and the liveness of the experience. Liveness means that an incremental action should result in an incremental result. This is possible without fluidity, which means that "taking the incremental action" may not be ergonomic for some reason.

(However, I will note that fluidity is SUPER important to me. I would love to build an interface that's only optionally dependent on the mouse.)

All Literals are GUIs...

If we throw out text-based coding and agree to a structured editor of some kind, you may realize that we can automatically represent colors as a color picker instead of (or in addition to) a hex value or rgb value. Cyrus Omar has work where he embed's a regex playground right into the IDE. (As Tudor Girba says, whenever you leave the IDE, the "I" has failed.)

If you follow this line of thinking, you realize that ALL literals are GUIs! Numbers can be scrubbable or many other interactive representations, booleans are a checkbox thing... We can even nest GUIs! You could have a string as a widget which is basically a text box but you can add other arbitrary expressions inside - no more escaping characters necessary! Lists can be a specialized GUI where you can add and remove expressions or do comprehensions.

We can continue this idea for all kinds of expressions: if-expressions, pattern-matching, lambdas, function application... Can we build nestable custom GUIs for every part of System F/Haskell?! I think so!

The key here is the nestability. Normally a color picker or other GUI is a top-level thing. But why? I don't see any reason why it can't be as expression-like as coding.

When you define a function it will default to a basic representation but it should allow you to add a more specific GUI to represent your function!

These thoughts were inspired mostly by Tudor Girba, but also are related to Niko Autio's Microeditor ideas.

I am curious how to combine this idea with the ideas of Hazel and Josh's principle of radical visibility of preview evaluation with test data. Same with the hashing stuff. There are so many cool PX ideas to combine together!

Drawing this out will be interesting. And if I can show how we can do any pattern from Haskell, completeness is guaranteed!

In terms of implantation, not sure if HTML or canvas is the way to go. Would be fun to play with Turbine on this project...

## Dec 13, 2018

https://futureofcoding.org/log#one-prototype-at-a-time

Now that I'm settled on Turbine (modulus the 15 issues I created, and many more to come), I'm beginning to think about what I hope to build with Turbine: p4.

I was getting a bit overwhelmed with all my ideas for p4. I made a big list and asked JE for help figuring out what's essential and what I can do without for this prototype:

Immutable editing
- content hash like Unison and IPFS
- do we hash definitions or expressions?
- do we do work to cannonicalize hashes? (such as commutativity of addition)
- no delete
- no edit, only create new thing
- refacorting
Detachable GUI Literals
- Everywhere you have a GUI, you need to be able to put an expression: detachable GUI literals!
- GUI for every system f language primitive
- Bootstrap visual editor?!
- Are all apps just expressions visualized?
- Expression-based operating system instead of file-based or app-based
Other ideas
- A la carte language semantics, like haskell langauge extensions
- Always running, like spreadsheet or google doc, no run or pause, only open or close
- infinite canvas (can I still use Turbine...?)
- exploring which concepts we can move from the language compiler to the editor (no free variables, type inference is just local if not just a type “suggestion”)

He helped me realize that most of these ideas represent entirely different prototypes! I can't just combine them all like this. Prototypes explore one new idea at a time. So what's the idea for p4?

JE hit the nail on the head: p4 is an IDE for Turbine. This is particularly funny because p1 was an IDE for JQuery and p2 was an IDE for VueJS! Those both used blockly but that wouldn't help with p4 because blocks solve a different problem than what p4 is trying to solve: improve the PX of DCTP, most directly by showing live streams as the user interacts with the output.

On my run a few hours ago, I was thinking about what my data model would be for p4. Then it hit me: Turbine is the data model. (Along with hareactive, io, and jabz.) Those stream data structures are the thing the p4 user is creating.

I could start p4 by building a projectional editor for Turbine/hareactive/io/jabz and all of JS. But can I get away with less?

One way to get away with less is to build a UI for the turbine/hareactive/io/jabz functions/combinators and then have users type regular text js where we need functional expressions (similar to aprt.us and pane).

The other way to do even less is to allow text to be the interaction model for Turbine and simply start with a devtools that shows the streams. If my main conviction is that seeing the streams will help, let's just do that. What's great about this idea is that a stream visualizer is a pre-req for any other p4 end-product anyways, so I might as well just publish it as it's own thing, and then move on afterwards.

In summary, p4 is now the Turbine devtools project, in the spirit of the CycleJS devtools.

Wait... because this feels reachable (I can imagine myself achieving it in early 2019 and it existing), it makes me wonder more about next steps...

Do I try to take up the Turbine mantel and get others to use it?
Do I go towards the structured editor idea?
Do I experiment with one of the other ideas listed above?

All valid answers. The thing is, whatever I do, I need a UI library I love, and that library needs a stream visualizer, so build that first. And then we'll talk.

Navigation Menu

Repository navigation

p4 #86

Infinite canvas drawings

Other tools for inspiration

Worst parts of Reflex/Haskell

p4 prioritized features

Implementation

p4 thinking for JE meeting

Theme of my work

Some responses about fluid Haskell

Biggest experience problems with Haskell/Reflex

1. Speed of feedback loop

2. API Discoverability

3. Plumbing code

4. Visualize streams

5. Direct manipulation

Next step ideas

1. Stay in Haskell

2. Build/use JS library/framework

3. Build/use JS Haskell interpreter

4. Build a higher-level system that compiles to JS

Fluidity vs structure

Conal -> Turbine

p4 thoughts 12/3/18

Start with non UI motivating problems

Fluidity is not the initial focus

All Literals are GUIs...

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

p4 #86

Description

Activity

stevekrouse commented on Nov 14, 2018

Infinite canvas drawings

Other tools for inspiration

Worst parts of Reflex/Haskell

p4 prioritized features

Implementation

stevekrouse commented on Dec 3, 2018

stevekrouse commented on Dec 3, 2018

p4 thinking for JE meeting

Theme of my work

Some responses about fluid Haskell

Biggest experience problems with Haskell/Reflex

1. Speed of feedback loop

2. API Discoverability

3. Plumbing code

4. Visualize streams

5. Direct manipulation

Next step ideas

1. Stay in Haskell

2. Build/use JS library/framework

3. Build/use JS Haskell interpreter

4. Build a higher-level system that compiles to JS

stevekrouse commented on Dec 3, 2018

stevekrouse commented on Dec 3, 2018

Fluidity vs structure

Conal -> Turbine

stevekrouse commented on Dec 3, 2018

p4 thoughts 12/3/18

Start with non UI motivating problems

Fluidity is not the initial focus

All Literals are GUIs...

stevekrouse commented on Dec 13, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions