Projectreunion: Discussion: What do you want in a better clipboard API for Windows?

[metathinker]

One part of Windows we at Microsoft (myself, @ShawnSteele, and some others) have been thinking of evolving as part of Project Reunion is the API for the Windows clipboard.

We've already seen a great idea for a clipboard improvement in issue #62 - allowing apps that were previously prohibited from accessing the clipboard in the background to do so when responding to a toast notification. As we think about whether and how to decouple the clipboard API and implementation into the eventual Project Reunion codebase, we'd like to hear some more ideas.

So, if you have pain points or improvement ideas for the clipboard that aren't quite fleshed out enough for an issue of their own, feel free to post them here! We'll try to keep up and listen, and may spin off issues for ideas as they get refined.

Related Links

62 #67

[hansmbakker]

Synchronization of the clipboard between phone and windows devices would be nice!

[metathinker]

Synchronization of the clipboard between phone and windows devices would be nice!

We actually have that now! It only works with Samsung Android phones for now, though.

If this feature were available to more phones and/or more Windows devices, how might you imagine programming it?

[metathinker]

Clipboard APIs today (wall of text follows)

For your reference, here's the current state of the Windows clipboard API from a very high level. As we look to craft a potential Project Reunion API for clipboard, we'd want to figure out both how to help you avoid traps that the current API presents and how to give you new features you could use to make your users happier.

There are 3 basic API surfaces for interacting with the Windows clipboard:

• The classic clipboard API, including GetClipboardData() and friends in windows.h.
  
  
  
  • This API thinks in terms of buffers of serialized data.
  
  
  • It allows source apps for clipboard data to defer rendering (serializing) data for certain data formats they claim to have copied.
  
  
  • Before clipboard source apps cleanly exit, Windows will force them to render delay-rendered data formats. (Specifically, this happens before the window is destroyed.)
  
  

• The OLE data transfer API, including OleGetClipboard() and friends in ole2.h.

  • This API thinks in terms of "IDataObject" objects. The source app for clipboard data must provide both the code of an IDataObject implementation and the actual data accessed through that object.
  • Source apps using this API always defer rendering data formats by default.
  • Windows doesn't force source apps to render all data formats before they exit. If the source app doesn't force itself to render all formats by "flushing" the clipboard, then unrendered data formats could be lost when it exits.

• Windows.ApplicationModel.DataTransfer.Clipboard and friends.

  • This API thinks in terms of "DataPackage" objects. Windows provides the DataPackage implementation; the source app only needs to bring the data.
  • Source apps can choose to defer rendering data on a per-format basis, as with the classic clipboard API.
  • "Flushing" the clipboard is a separate concept from deferred rendering, unlike with OLE data transfer. If the source app doesn't "flush" the clipboard, then after it exits, data for _both_ rendered and unrendered formats could be lost.

All 3 APIs are equivalent in basic clipboard functions, but Windows.ApplicationModel.DataTransfer also recently added programmatic access to the clipboard history and cloud-roamed clipboard data, as part of the addition of those features in the November 2018 update to Windows (Windows 10 version 1809).

All 3 APIs have their own advantages and pitfalls - as alluded to above, both the OLE data transfer API and Windows.ApplicationModel.DataTransfer potentially make it too easy for copied data to disappear from the clipboard when source apps are closed, frustrating users when they try to paste text from their web browser or email program and getting nothing instead, or a previously copied item they no longer want.

[memexy]

This discussion was posted on hacker news. I posted a comment there but I'm posting it here as well to make sure you folks see it.

Automatic saving to a specified location and OCR. The current clipboard doesn't support automatic saving so I have to use another program to do that.

[mdtauk]

I just want my copy and paste actions to work every time, without fail, and without complication

[metathinker]

I just want my copy and paste actions to work every time, without fail, and without complication

In our observation at Microsoft, a lot of problems with clipboard reliability are caused by apps that use the clipboard APIs in the "wrong" way. For example:

• The classic clipboard API allows any app that is reading or writing clipboard data to accidentally lock out all other apps from accessing the clipboard - to do that, you can just forget to call CloseClipboard() when you're done. The most obvious symptom of this is a Paste operation pasting an older image or piece of text, as the most recent cut or copy attempt had silently failed due to the clipboard lockout.

• As I noted above, the OLE data transfer API and Windows.ApplicationModel.DataTransfer require an explicit "flush clipboard" step. If a source app for copied clipboard data forgets to flush its written data values, they could be lost when the app exits, leading to no data being pasted on paste attempts.

To get to 100% clipboard reliability, we'll need to figure out how to fix our mistakes that have led people down the wrong paths. That's part of the reason I started this discussion - we'd like to learn how to make it easy for people to do the right thing and how to make it hard or impossible for them to do the wrong thing.

[hansmbakker]

I don't know about the feasibility / effort required of different options, but would it be possible to have a Roslyn Analyzer for that to point at mistakes in the use of these APIs (as a short term helper)?

I can imagine that changing the APIs themselves could be a longer-term project.

[metathinker]

would it be possible to have a Roslyn Analyzer for that to point at mistakes in the use of these APIs (as a short term helper)?

That's an interesting idea. From my understanding, Roslyn analyzers in particular only work for C# and VB, but other .NET managed code languages, as well as native code, could benefit from similar static analysis. I don't think we at Microsoft have really used static code analysis in Windows native code development for cases other than "this could be a security vulnerability", but perhaps we should consider broadening our view...

[sylveon]

I really want an API that allows classic & UWP app to set data it sends to the clipboard as "private." This will be useful for password manager apps, where I copy the password in the clipboard and then paste it into a web browser or other app. Currently, this goes right into the clipboard history and is then promptly synced on other devices.

I do not want my passwords in the clipboard history for the simple reason that somebody with access to the computer could press Win-V and view passwords I used in the past.
I also do not want my passwords to be synced to any remote server, (Microsoft or not) for obvious privacy and data security principles.

[sylveon]

Not sure how I managed to do that just by posting a comment, but ok

[metathinker]

I really want an API that allows classic & UWP app to set data it sends to the clipboard as "private." This will be useful for password manager apps, where I copy the password in the clipboard and then paste it into a web browser or other app. Currently, this goes right into the clipboard history and is then promptly synced on other devices.

I do not want my passwords in the clipboard history for the simple reason that somebody with access to the computer could press Win-V and view passwords I used in the past.
I also do not want my passwords to be synced to any remote server, (Microsoft or not) for obvious privacy and data security principles.

There are already 2 ways to prevent newly set clipboard data values from being synchronized by cloud clipboard or entered into the local clipboard history:

1. If you use Windows.ApplicationModel.DataTransfer, call Clipboard.SetContentWithOptions() and pass a ClipboardContentOptions record with IsRoamable and IsAllowedInHistory both set to false.
2. If you use the classic clipboard API or OLE data transfer, set the custom clipboard format ExcludeClipboardContentFromMonitorProcessing to any data value.

Are neither of those options working for you?

[sylveon]

I wasn't aware of those APIs, I just based off my assumption on the behavior of my current password manager. I'll go bother them with this then 😅

[MuleaneEve]

For privacy reasons, I think that every (approved) access to the clipboard should trigger the appearance of an icon in the taskbar (just like with geolocation access).

An extra step that I would also like to see for all these sensitive services is that there should be a way to see a history of these accesses after the fact. That way, we can audit apps to confirm that they did not abusing these services while we were distracted.

[Poopooracoocoo]

Synchronization of the clipboard between phone and windows devices would be nice!

We actually have that now! It only works with Samsung Android phones for now, though.

If this feature were available to more phones and/or more Windows devices, how might you imagine programming it?

There's also Pushbullet (off topic but it's worth mentioning alternatives to Your Phone, which is quite limited in the devices it supports)