We know that over the years Qt Quick's multi-touch support has been inadequate in many of its use cases. We have a PinchArea , to handle zooming, rotating and dragging with two fingers; and MultiPointTouchArea , which can be used to show some sort of interactive feedback for the touch points, or maybe you could write a state machine in JavaScript to recognize some kind of gesture. As for the rest of Qt Quick, the main issues are: 1) support for mouse events; 2) Qt assumes that there is only one mouse (the "primary mouse pointer"); 3) QMouseEvent and QTouchEvent (and a few more) do not have a suitable intermediate base class, so they are delivered independently of each other; 4), which was tricky early on to treat touch events as mouse events and deliver them the same way. So the result is that you can't interact with two MouseAreas or Flickables at the same time, e.g. It also means that you cannot press two buttons at the same time or drag two sliders at the same time if they are implemented with MouseArea.
At first I was hoping to fix this by making MouseArea and Flickable separate touch events. The patches for this were pretty complex, and added a lot of duplicated logic for a full parallel delivery path: QMouseEvent would take one path and QTouchEvent the other, hoping the interaction would work as best as possible. It was months of work, and at the end it mostly worked... but it was difficult to keep passing all the existing autotests, and colleagues were concerned that this was a behavioral change. MouseArea, proclaimed by its name, handles mouse events, so once it started handling touch events separately, its name became misused. All of a sudden, you could press two buttons, tabs, or radio buttons at the same time in apps and control sets that weren't designed for it. (So we tried adding a bool property that could be set, but having to set it on every MouseArea would be ugly.) MouseArea and Flickable also need to interact a lot, so changes had to be made together to make it work. This was possible but was not added in Qt 5.5 due to vagueness.
So in the end we chose a different path, after we found a reasonable combination of the proposed ideas.
One idea was that since we can't (yet can't) refactor the QEvent hierarchy due to the binary compatibility mandate, we could instead create wrapper classes that make events look like what we want , complete with properties for the benefit of QML, and deliver these wrapper classes instead of regular ones using a single event delivery path to QQuickWindow and QQuickItem.
Another idea was to realize that dynamically creating and destroying these wrapper events is stupid: instead, we began to use a pool of instances, as we did in other cases when the "event" object is emitted by a signal (for example, the object emitted by MouseArea .positionChanged is always the same instance since Qt 5.8). With this optimization, pushing one event to another is no longer a big performance hit.
Another idea was raised: maybe it would be nice if handling the pointing device event was as simple as using the keys the property is bound to: for example, Mouse.onClicked: {...} or PointingDevice.onTapped: {...} But soon after that came the realization that there can only be one instance of an attached property per element it's attached to. One of the problems with MouseArea is that it tries to do too much, so it doesn't make sense to just re-implement all the functionality in a monolithic MouseAttached . We wanted to be able to handle a click or tap without caring what device it came from, because that's what every Button control needs to do. The same goes for any gesture that can be performed with a mouse or one finger. Perhaps there should be one attached property per gesture rather than one per device type?
Since QML is a declarative language, it's nice to be able to declare constraints rather than writing if/else statements in JavaScript callbacks. If the object that handles the events is designed so that it doesn't care which device the gesture came from, there will still be cases where your application will take care of it: you want to perform different actions depending on whether it's pressed on the touch screen or right-clicked, or you want to do something else if the control key is held down while dragging an object. By providing multiple instances of these handler objects, we can declare constraints on them by setting properties. It should be good to have as many cases as you need. Each instance should be lightweight so that you are not afraid to have too many instances. The implementation should be in C++ and should be simple and clear. Each handler should do one thing, or at most a few very close things, and do them well.
For now, these issues have distracted us from the idea of using attached properties. Instead, we have a new family of classes in Qt Quick: pointer handlers.
A pointer handler is a type of object that you can declare inside any element that handles events from pointing devices on behalf of that element. You can declare as many of them as you like, usually one for each interaction scenario. All the general restrictions we could think of are declarative: you can make a handler only react if it's a touch event, only for certain mouse buttons, only if the correct number of fingers are pressed within an element, only if a special modifier key etc.
Draggable balls
Regular gestures are represented by their own handler types. For example, if you declare
Rectangle {
width: 50; height: 50; color: "green"
DragHandler { }
}
then you have a rectangle that can be dragged around the stage with mouse or touch, without writing Javascript and even without having to bind the handler to its parents. It has a target property and the default value is its parent. (But by setting the target to another element, you can capture events within one element but manipulate the other.)
Of course, if you have these two green rectangles with DragHandlers inside, you can drag them both at the same time with different fingers.
Each pointer handler is a QObject, but it's not a QQuickItem and it doesn't have too many variables of its own, so each instance is about as small as a QObject subclass.
Each single point handler has a point property to provide all the details we can find about the point of touch or mouse click. There are pressure and ellipseDiameters properties: some devices may have force sensors, while others may measure contact path size (but many devices do not provide this). Such devices have a velocity property that is guaranteed to be determined: we calculate and average the speed over the last few movements for a slightly smoother response. Having the speed available can potentially activate speed-sensitive gestures: perhaps the click needs to be performed at a certain minimum speed. (Is this the correct way to tell a click from a drag? It wasn't easy to make that distinction before.) Or, if you have velocity while you release, the drag gesture can end up with momentum: the object moves a short distance in one direction. It makes your interface more alive. So far, we haven't formalized MomentumAnimation into a supported animation type, but there is a pure-QML prototype in tests/manual/pointer/content/MomentumAnimation.qml.
Pox
TapHandler handles all tap and release gestures: single quick taps or clicks, double taps, other number clicks, holding a tap for a configurable period of time, or holding it for various periods of time in all unique ways . (When you touch the touch screen, it often doesn't make any sound, but you can press the mouse button, so we thought "tap" is a more reliable future for this gesture than "click".) You can show feedback in proportion to how long the click is (an expanding circle, a progress bar, or something like that).
Pinch me if it's true
There is PinchHandler . If you declare it inside an element, you can scale, rotate, and drag that object with pinch gestures. You can scale any part of the element you like (improved in PinchArea). It can also handle more fingers: you can declare PinchHandler {minimumTouchPoints: 3} to require a three-finger pinch gesture. All transformations are then relative to the center point between the three fingers, and scaling refers to the average increase or decrease in the spread between them. The idea came from the fact that some versions of Ubuntu use a three-finger pinch to manipulate windows: apparently they thought that the contents in a window could be used for two-finger gestures, but most applications don't use three fingers, so it's okay to reserve the three-finger pinch for zooming and moving windows around the desktop. Now, since you can write a Wayland composer in QML, you can easily recreate that experience.
Getting to the points
Finally, there is PointHandler . Unlike the others, it doesn't manipulate its target element: it only exists to provide a point property. It is similar to a standalone TouchPoint in a MultiPointTouchArea and can be used for the same purpose: to provide interactive feedback as touch points or as a mouse cursor moving around the scene. Unlike MultiPointTouchArea, it captures touch points or the mouse non-exclusively, so having this interactive feedback does not prevent interaction with other handlers on other elements at the same time. The animation on this page uses it to make the finger sprites follow my fingers.
Is that all?
So, I'll now move on to the reasons why this stuff is in the Tech Preview at 5.10. One reason is that it's incomplete: we still lack support for mouseover, mouse wheel, and tablet styluses (for now, the stylus is still treated as a mouse). None of the handlers are speed sensitive. We can imagine a few more handlers that could be written. There should be an open C++ API so you can create your own. Handlers and Flickable get along a bit, but Flickable is a complex monolithic Item and we think it could be refactored later. There is a manual FakeFlickable test that shows how you can recreate most of your functionality in QML with two regular elements, plus a DragHandler and a few animations.
Another reason is naming. "Pointer handlers" sounds ok in isolation, but there is pre-existing terminology that makes it confusing: a pointer can be a variable pointing to a memory location (but that's not what we mean here), and a handler can be a reverse function sort call you write in JavaScript. If you write TapHandler {onActiveChanged: ...} then are you saying your handler has a handler? We could use the word "callback" instead, but that's an anachronism in some circles, and in QML our habits are hard to change now.
Another reason is that QtLocation has some complex use cases that we want to use as an example to prove that it is possible to move the map (and any interactive content on it) with a bit of readable code.
Maybe I'll follow this up later with another post about a different way of recognizing gestures and how that might affect how we think about pointer handlers in the future. I haven't explained the concept of passive capture yet. There is also more information on how to create components that internally use pointer handlers, but the copyright author can override the behavior if necessary.
So right now pointer handlers are in the Tech Preview in 5.10. We hope you enjoy playing with them for a long time. Especially if you have some old frustrations about interaction scenarios that weren't possible before. Moreover, if you ever wanted to create a touch user interface (perhaps even multi-user?) without writing C++ event-forwarding and QObject-event-filtering dirt. We need to start getting feedback from everyone with unique use cases while we have the freedom to make big changes if needed to make things easier for you.
So far, the use cases have mostly been in tests/manual/pointer . The test source is not included in the release build package, so to try them out you need to download the Qt source packages or get them from the [git] repository(http://code.qt.io/cgit/qt/qtdeclarative.git/tree /tests/manual/pointer/) . Some of them can be turned into examples for upcoming releases.
Since this is in technical preview, the implementation will continue to improve during the 5.10 series, so follow the 5.10 git branch to keep up with the latest features and bug fixes.
Article written by: Shawn Rutledge | Thursday, November 23, 2017
