SMITH070+
`
`USER INTERFACE SYSTEM, METHOD, AND COMPUTER PROGRAM
`PRODUCT
`
`CROSS-REFERENCE TO RELATED APPLICATIONS
`
`[0001] If any definitions (e.g. figure reference signs, specialized terms, examples,
`
`data, information, etc.) from any related material (e.g. parent application, other related
`application, material incorporated by reference, material cited, extrinsic reference, etc.)
`conflict with this application (e.g. abstract, description, summary, claims, etc.) for any
`purpose (e.g. prosecution, claim support, claim interpretation, claim construction, etc.),
`
`then the definitions in this application shall apply.
`
`BACKGROUND AND FIELD OF INVENTION
`
`[0002] Embodiments of the present invention generally relate to consumer electronic
`devices, particularly cell phones, tablets, and other mobile devices (however,
`
`applicability is also relevant to non-mobile devices).
`
`Apple Inc. v. Smith Interface Techs., LLC
`IPR2024-01091 | Smith EX2010 | Page 1 of 194
`
`
`
`
`
`
`
`
`BRIEF SUMMARY
`
`[0003] A system, method, and computer program product are provided for a touch or
`pressure signal-based interface. In operation, a touch or pressure signal is received in
`association with a touch interface of a device. To this end, a user experience is altered,
`utilizing the signal. In different embodiments, various features may be further
`incorporated in association with the system, method, and computer program product, for
`
`improvement purposes.
`
`Page 2 of 194
`
`
`
`
`
`
`
`
`-3-
`
`BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
`
`[0004] So that the features of various embodiments of the present invention can be
`understood, a more detailed description, briefly summarized above, may be had by
`reference to various embodiments, some of which are illustrated in the accompanying
`drawings. It is to be noted, however, that the accompanying drawings illustrate only
`embodiments and are therefore not to be considered limiting of the scope of various
`embodiments of the invention, for the invention may admit to other effective
`embodiments. The following detailed description makes reference to the accompanying
`
`drawings that are now briefly described.
`
`[0005] Figure 1 illustrates a network architecture, in accordance with one
`embodiment.
`[0006] Figure 2 shows a representative hardware environment that may be associated
`
`with the servers and/or clients of Figure 1, in accordance with one embodiment.
`
`[0007] Figure 3 shows a method for altering a user experience based on a received
`
`signal, in accordance with one embodiment.
`
`[0008] Figure 4 shows a method for defining a selection made within a user interface
`
`based in part on contact pressure, in accordance with one embodiment.
`
`[0009] Figure 5 shows a pressure-sensitive user interface for making a selection, in
`
`accordance with one embodiment.
`
`[0010] Figure 6 shows a method for determining the context of a contact pressure-
`based selection and choosing an appropriate selection function, in accordance with one
`
`embodiment.
`
`[0011] Figure 7 shows a device having a backtouch interface, in accordance with one
`
`embodiment.
`
`[0012] Figure 8 shows a method for providing feedback to the user of a backtouch
`
`interface, in accordance with one embodiment.
`
`Page 3 of 194
`
`
`
`
`
`
`
`
`-4-
`
`[0013] Figure 9 shows a pressure-sensitive user interface for making a selection using
`
`a backtouch interface, in accordance with one embodiment.
`
`[0014] Figure 10 shows a user interface for defining settings associated with a
`
`backtouch interface, in accordance with one embodiment.
`
`[0015] Figure 11 shows a user interface for defining settings associated with a
`
`pressure-sensitive interface, in accordance with one embodiment.
`
`[0016] Figure 12 shows a method for assisting a user in defining touch states, in
`
`accordance with one embodiment.
`
`[0017] Figure 13 shows a user interface for assisting a user in defining touch states,
`
`in accordance with one embodiment.
`
`[0018] Figure 14 shows a user interface for indicating that a backtouch or pressure-
`
`sensitive interface is activated, in accordance with one embodiment.
`
`[0019] Figure 15 shows a user interface for defining a backtouch feedback style, in
`
`accordance with one embodiment.
`
`[0020] Figure 16 shows an alternative method for defining a selection made within a
`
`user interface based in part on contact pressure, in accordance with one embodiment.
`
`[0021] Figure 17 shows a user interface for performing operations on a selection, in
`
`accordance with one embodiment.
`
`[0022] Figure 18 shows a method for utilizing contact pressure-based gestures, in
`
`accordance with one embodiment.
`
`[0023] Figure 19 shows an example of a contact pressure-based gesture for scrolling
`
`a text field, in accordance with one embodiment.
`
`[0024] Figure 20 shows an example of a multitouch pressure gesture for indicating a
`
`direction, in accordance with one embodiment.
`
`Page 4 of 194
`
`
`
`
`
`
`
`
`5-
`
`[0025] Figure 21 shows an example of a multitouch pressure gesture for indicating a
`
`rotation, in accordance with one embodiment.
`
`[0026] Figure 22 shows an example of a front-back gesture, in accordance with one
`embodiment.
`[0027] Figure 23 shows an example of a virtual control knob gesture, in accordance
`
`with one embodiment.
`
`[0028] Figure 24 shows a method for gesture differentiation, in accordance with one
`embodiment.
`[0029] Figure 25 shows an example of an “anchor and move” gesture, in accordance
`
`with one embodiment.
`
`[0030] Figure 26 shows an example of a twist gesture, in accordance with one
`embodiment.
`[0031] Figure 27 shows a method for determining touch states using a touch-sensitive
`
`interface, in accordance with one embodiment.
`
`[0032] Figure 28 shows examples of pattern-based gestures, in accordance with one
`
`embodiment.
`
`[0033] Figure 29 shows file sharing triggered by a gesture, in accordance with one
`
`embodiment.
`
`[0034] Figure 30 shows an example of touch-based device authentication, in
`
`accordance with one embodiment.
`
`[0035] Figure 31 shows an example of multi-device facial recognition, in accordance
`
`with one embodiment.
`
`[0036] Figure 32 shows two displays combined to function as one, in accordance
`
`with one embodiment.
`
`Page 5 of 194
`
`
`
`
`
`
`
`
`-6-
`
`[0037] Figure 33 shows a user interface for defining custom gestures, in accordance
`
`with one embodiment.
`
`[0038] Figure 34 shows a user interface for defining pressure inputs for a custom
`
`gesture, in accordance with one embodiment.
`
`[0039] Figure 35 shows a user interface for defining touch inputs for a custom
`
`gesture, in accordance with one embodiment.
`
`[0040] Figure 36 shows a user interface for defining surface inputs for a custom
`
`gesture, in accordance with one embodiment.
`
`[0041] Figure 37 shows a user interface for summarizing contact inputs for a custom
`
`gesture, in accordance with one embodiment.
`
`[0042] Figure 38 shows a user interface for defining system inputs for a custom
`
`gesture, in accordance with one embodiment.
`
`[0043] Figure 39 shows a pressure-sensitive user interface containing a virtual
`
`trackball, in accordance with one embodiment.
`
`[0044] Figure 40 shows a device with slide pads, in accordance with one
`embodiment.
`[0045] Figure 41 shows an example of a slide pad gesture for performing an auxiliary
`
`function, in accordance with one embodiment.
`
`[0046] Figure 42 shows a user interface for providing quick access to favorite
`
`operations, in accordance with one embodiment.
`
`[0047] Figure 43 shows a device with a non-uniform display, in accordance with one
`embodiment.
`[0048] Figure 44 shows a light valve technique, in accordance with one embodiment.
`
`Page 6 of 194
`
`
`
`
`
`
`
`
`-7-
`
`[0049] Figure 45 shows a device display mixing color pixels with black and white
`
`pixels, in accordance with one embodiment.
`
`[0050] Figure 46 shows a device display mixing color with black and white across a
`
`matrix, in accordance with one embodiment.
`[0051] Figure 47 shows a modulated display, in accordance with one embodiment.
`
`[0052] Figure 48 shows a device with a display having a non-uniform backlight, in
`
`accordance with one embodiment.
`
`[0053] Figure 49 shows a device having a removable back sensor or display, in
`
`accordance with one embodiment.
`
`[0054] Figure 50 shows a user interface containing an adaptive soft keyboard, in
`
`accordance with one embodiment.
`
`[0055] Figure 51 shows a user interface with context-sensitive backlighting, in
`
`accordance with one embodiment.
`
`[0056] Figure 52 shows a 3D layered user interface, in accordance with one
`
`embodiment.
`
`[0057] Figure 53 shows a device which may be converted into a 3D layer enabled
`
`device, in accordance with one embodiment.
`
`[0058] Figure 54 shows a 3D layered user interface for augmented reality, in
`
`accordance with one embodiment.
`
`[0059] Figure 55 shows a device with user gaze tracking sensors, in accordance with
`
`one embodiment.
`
`[0060] Figure 56 shows the use of eye tracking to generate 3D depth cues, in
`
`accordance with one embodiment.
`
`[0061] Figure 57 shows a layered display, in accordance with one embodiment.
`
`Page 7 of 194
`
`
`
`
`
`
`
`
`-8-
`
`[0062] Figure 58 shows an automultoscopic display, in accordance with one
`
`embodiment.
`
`[0063] Figure 59 shows a layered display containing depth fused 3D images, in
`
`accordance with one embodiment.
`[0064] Figure 60 shows a light field camera, in accordance with one embodiment.
`
`[0065] Figure 61 shows a 3D layered user interface with a floating soft keyboard, in
`
`accordance with one embodiment.
`
`[0066] Figure 62 shows a backtouch enabled 3D layered user interface, in accordance
`
`with one embodiment.
`
`[0067] Figure 63 shows a tactile feedback enabled display, in accordance with one
`
`embodiment.
`
`[0068] Figure 64 shows a watch-based display extension, in accordance with one
`
`embodiment.
`
`[0069] Figure 65 shows a wireless storage ring and key, in accordance with one
`
`embodiment.
`
`[0070] Figure 66 shows a 3D layered user interface having a clipboard layer, in
`
`accordance with one embodiment.
`
`[0071] Figure 67 shows a 3D layered user interface having an operation layer, in
`
`accordance with one embodiment.
`
`[0072] Figure 68 shows a user interface for providing touch feedback, in accordance
`
`with one embodiment.
`
`[0073] Figure 69 shows a user interface for providing increased security, in
`
`accordance with one embodiment.
`
`Page 8 of 194
`
`
`
`
`
`
`
`
`-9-
`
`[0074] Figure 70 shows a user interface for presenting a contextual menu, in
`
`accordance with one embodiment.
`
`Page 9 of 194
`
`
`
`
`
`
`
`
`-10-
`
`DETAILED DESCRIPTION
`
`[0075] Figure 1 illustrates a network architecture 100, in accordance with one
`
`embodiment. As shown, a plurality of networks 102 is provided. In the context of the
`present network architecture 100, the networks 102 may each take any form including,
`but not limited to a local area network (LAN), a wireless network, a wide area network
`
`(WAN) such as the Internet, peer-to-peer network, etc.
`
`[0076] Coupled to the networks 102 are servers 104 which are capable of
`communicating over the networks 102. Also coupled to the networks 102 and the servers
`104 is a plurality of clients 106. Such servers 104 and/or clients 106 may each include a
`desktop computer, lap-top computer, hand-held computer, mobile phone, personal digital
`assistant (PDA), tablet computer, peripheral (e.g. printer, etc.), any component of a
`computer, and/or any other type of logic. In order to facilitate communication among the
`
`networks 102, at least one gateway 108 is optionally coupled therebetween.
`
`[0077] Figure 2 shows a representative hardware environment that may be associated
`with the servers 104 and/or clients 106 of Figure 1, in accordance with one embodiment.
`Such figure illustrates a typical hardware configuration of a mobile device in accordance
`with one embodiment having a central processing unit 210, such as a microprocessor, and
`
`a number of other units interconnected via a system bus 212.
`
`[0078] The mobile device shown in Figure 2 includes a Random Access Memory
`(RAM) 214, Read Only Memory (ROM) 216, an I/O adapter 218 for connecting
`peripheral devices such as disk storage units 220 to the bus 212, a user interface adapter
`222 for connecting a keyboard 224, a mouse 226, a speaker 228, a microphone 232,
`and/or other user interface devices such as a touch screen (not shown) to the bus 212,
`communication adapter 234 for connecting the mobile device to a communication
`network 235 (e.g., a data processing network) and a display adapter 236 for connecting
`
`the bus 212 to a display device 238.
`
`Page 10 of 194
`
`
`
`
`
`
`
`
`-l|1-
`
`[0079] The mobile device may have resident thereon any desired operating system. It
`will be appreciated that an embodiment may also be implemented on platforms and
`operating systems other than those mentioned. One embodiment may be written using
`JAVA, C, Objective C, and/or C++ language, or other programming languages, along
`with an object oriented programming methodology. Object oriented programming (OOP)
`
`has become increasingly used to develop complex applications.
`
`[0080] Of course, the various embodiments set forth herein may be implemented
`utilizing hardware, software, or any desired combination thereof. For that matter, any
`type of logic may be utilized which is capable of implementing the various functionality
`
`set forth herein.
`
`[0081] Figure 3 shows a method 300, in accordance with one embodiment. As an
`option, the method may be implemented in the context of the architecture and
`environment of any subsequent Figure(s). Of course, however, the method may be
`
`implemented in any desired environment.
`
`[0082] As shown in operation 302, a signal is received in association with a touch
`interface of a device. In one embodiment, the signal may include a pressure signal which
`is indicative of a magnitude of pressure. In the context of the present description, such
`pressure signal may include any signal that is a function of or related to a pressure
`applied to the device. In one embodiment, such pressure signal may be indicative of a
`magnitude of pressure being applied to a touch interface of the device. In various
`embodiments, such pressure signal may be generated by a pressure sensor including, but
`not limited to those described in the context of subsequently described embodiments, or
`
`any other mechanism, for that matter, that is capable of generating the pressure signal.
`
`[0083] In other embodiments, the signal may include a touch signal indicative of a
`touch interface being touched. In the context of the present description, such touch signal
`may or may not be indicative of a magnitude of pressure being applied to a touch
`
`interface of the device. For example, such touch signal may, in one embodiment, simply
`
`Page 11 of 194
`
`
`
`
`
`
`
`
`-12-
`
`indicate whether applied pressure is sensed, or not, e.g. not necessarily gauge any more
`
`than two pressure states, including pressure-sensed and pressure-not-sensed, etc.
`
`[0084] Also in the context of the present description, the aforementioned device may
`include any device including, but not limited to those described in the context of this
`and/or subsequently described embodiments. Further, in various embodiments, the touch
`interface may or may not be combined with a display. For example, if the touch interface
`is combined with a display, the touch interface may include a touch screen. Thus, the
`touch interface may, in various embodiments, include, but is not limited to a touch screen
`or any other interface responsive to touch that is described in the context of this and/or
`
`subsequently described embodiments.
`
`[0085] In one optional embodiment, the touch interface may be positioned on a first
`face of a device that includes a display on a second face thereof. Further, the first face
`and second face may include any respective faces (e.g. front, back, left side, right side,
`top side, bottom side, etc.) that are different. Just by way of example, in various
`embodiments, the first face and the second face may be on opposite faces of the device of
`the device. Further, the first face may include a side face and the second face may
`include a front face of the device. Even still, the first face may include a back face and
`the second face may include a front face of the device. Of course, the touch interface
`
`may be positioned on the same face as the display, in other embodiments.
`
`[0086] As indicated in operation 304, a user experience may be altered, utilizing the
`signal. In the context of the present description, the user experience may include any
`aspect of the device that is capable of being directly or indirectly experienced by a user
`including, but not limited to those described in the context of this and/or subsequently
`
`described embodiments.
`
`[0087] For example, in the context of an embodiment involving the pressure signal,
`the user experience may be altered as a function of the magnitude of the pressure being
`applied to the touch interface, utilizing the pressure signal. Just by way of example, in
`various embodiments, the user experience may be altered by causing input to the device,
`
`by causing output from the device, by causing processing by the device, etc. In other
`
`Page 12 of 194
`
`
`
`
`
`
`
`
`-13-
`
`embodiments involving a touch signal, the user experience may be altered in any manner,
`
`utilizing the touch signal (dependent or independent of any fluctuation of pressure).
`
`[0088] Of course, the foregoing embodiments may be altered by adding and/or
`removing various features. For example, in one embodiment, the pressure signal may be
`received which is indicative of the magnitude of pressure being applied to a touch
`interface positioned on a first face of a device that includes a display on a second face
`thereof. To this end, the user experience may be altered as a function of the magnitude of
`
`the pressure being applied to the touch interface, utilizing the pressure signal.
`
`[0089] In another embodiment, a pressure signal may be received which is indicative
`of a magnitude of pressure being applied to a touch screen. To this end, the user
`experience may be altered as a function of the magnitude of the pressure being applied to
`the touch screen, utilizing the pressure signal. In still yet another optional embodiment, a
`touch signal may be received which is indicative of a touch interface being touched,
`where the touch interface is positioned on a first face of a device that includes a display
`on a second face thereof. To this end, the user experience may be altered, utilizing the
`touch signal. Again, any of the features described above (or hereinafter, for that matter)
`
`may or may not be combined in any desired manner.
`
`[0090] More illustrative information will now be set forth regarding various optional
`architectures and features with which the foregoing techniques discussed in the context of
`any of the present or previous figure(s) may or may not be implemented, per the desires
`of the user. For instance, various optional examples and/or options associated with the
`operations 302 and/or 304, and/or other optional features have been and will be set forth
`in the context of a variety of possible embodiments. It should be strongly noted,
`however, that such information is set forth for illustrative purposes and should not be
`construed as limiting in any manner. Any of such features may be optionally
`
`incorporated with or without the inclusion of other features described.
`
`[0091] Figure 4 shows a method 400 for defining a selection made within a user
`interface based in part on contact pressure, in accordance with one embodiment. As an
`
`option, the method 400 may be implemented in the context of the architecture and
`
`Page 13 of 194
`
`
`
`
`
`
`
`
`-14-
`
`environment of the previous Figures or any subsequent Figure(s). Of course, however,
`the method 400 may be carried out in any desired environment. It should also be noted
`
`that the aforementioned definitions may apply during the present description.
`
`[0092] As shown, it is determined whether sufficient initial contact pressure is being
`
`exerted on an interaction surface. See determination 402.
`
`[0093] In the context of the present description, an interaction surface refers to a
`surface through which a user may interact with a device. It may take up the majority of a
`device face, or just a subset. In various embodiments, an interaction surface may be
`sensitive to one or more types of interaction, including but not limited to, contact,
`pressure, and/or proximity. In one embodiment, an interaction surface is receptive to
`multitouch input. In another embodiment, an interaction surface may be non-planar. In
`yet another embodiment, an interaction surface may be transparent, and combined with a
`display. In still another embodiment, an interaction surface may be replaceable. Some
`embodiments of an interaction surface may have more than one texture, to provide tactile
`guides to a user when used on the back face of a device. Other embodiments of an
`
`interaction surface are non-rectangular.
`
`[0094] In various embodiments, an interaction surface may utilize one or more forms
`of technology to sense contact. These contact-sensing technologies may include, but are
`not limited to, capacitive, resistive, optical, surface acoustic wave based, and/or any other
`
`contact sensing technologies now known or later developed.
`
`[0095] In various embodiments, an interaction surface may utilize one or more forms
`of technology to sense proximity. These proximity-sensing technologies may include,
`but are not limited to, capacitive, resistive, eddy current, optical, ultrasonic, heat, electric
`
`field based, and/or any other contact sensing technologies now known or later developed.
`
`[0096] In various embodiments, an interaction surface may utilize one or more forms
`of technology to sense pressure. These pressure-sensing technologies may include, but
`are not limited to, conductive, resistive, piezoelectric, and/or any other pressure sensing
`
`technologies now known or later developed. In some embodiments, an interaction
`
`Page 14 of 194
`
`
`
`
`
`
`
`
`-15-
`
`surface may be able to only detect the total pressure being exerted on the surface. In
`other embodiments, an interaction surface may be able to discern the contact pressures
`
`associated with one or more points or areas of contact.
`
`[0097] In the context of the present description, contact pressure is the pressure
`associated with one or more contact points or contact areas, the pressure being exerted on
`an interaction surface. For example, in one embodiment, contact pressure may be the
`
`pressure exerted on an interaction surface by a single finger press.
`
`[0098] In the context of the present description, a contact area refers to the area of an
`interaction surface which is in physical contact with an implement of interaction. In
`various embodiments, the contact area may be described, stored, and processed, as a
`collection of cells that are part of an array. In one embodiment, this array may directly
`correspond to pixels of a display. In another embodiment, the array may correspond to a
`grid of discrete areas on an interaction surface which sense contact, pressure, or both. In
`still another embodiment, the contact area is represented as a primitive shape (e.g. circle,
`
`square, etc.) which best fits the actual area in contact with an implement of interaction.
`
`[0099] In the context of the present description, an implement of interaction refers to
`an object which is detectable by an interaction surface in one or more ways (e.g.
`proximity, contact, pressure, etc.). In various embodiments, implements of interaction
`may include, but are not limited to, fingers, skin, a stylus, and/or any other object with
`
`which a user may interact with a device.
`
`[00100] In the context of the present description, a contact point refers to a single
`location on an interaction surface which is associated with physical contact with an
`implement of interaction. In various embodiments, the contact point may be determined
`from a contact area. For example, in one embodiment, a contact point may be located in
`the center of its associated contact area. In another embodiment, a contact point may be
`
`located on an edge of its associated contact area.
`
`[00101] Determining whether sufficient initial contact pressure is being exerted on an
`
`interaction surface depends on a threshold contact pressure. In some embodiments, the
`
`Page 15 of 194
`
`
`
`
`
`
`
`
`-16-
`
`smallest degree of contact pressure may be sufficient, such that the definition of a
`selection may be triggered by any contact. In other embodiments, the definition of a
`selection does not occur until a non-negligible threshold contact pressure has been
`detected. In one embodiment, this threshold contact pressure may be defined by the user.
`
`In this way, accidental definitions of a selection may be avoided.
`
`[00102] In some embodiments, determination 402 may be performed only after certain
`events have occurred. For example, in one embodiment, the determination may be made
`only after receipt of an input or combination of inputs. Possible inputs include, but are
`not limited to, object proximity to the interaction surface, activation of a hardware switch
`or button, activation of a software switch or button, and/or any other input capable of
`being detected. In one embodiment, the determination may not be made until the host
`device is woken from a sleep state. In another embodiment, the determination itself may
`
`also wake the host device from a sleep state.
`
`[00103] If the result of determination 402 is that there is sufficient initial contact
`pressure being exerted on an interaction surface, the current contact pressure level is
`
`determined. See operation 404.
`
`[00104] Inthe context of the present description, a contact pressure level is the
`pressure reading generated from the raw data received from a touch sensor. In various
`embodiments, the contact pressure level may be represented by a number. For example,
`in some embodiments, the contact pressure level may be represented by an integer. In
`another embodiment, the contact pressure level may be represented by a floating-point
`number. In various embodiments, the contact pressure level may be represented by a
`percentage of the measurable range of contact pressures. In one embodiment, the contact
`pressure level may be represented by a unit less number. In another embodiment, the
`contact pressure level may be represented by an actual measure of the contact pressure,
`
`associated with a unit of pressure.
`
`[00105] In various embodiments, the contact pressure level may be represented by a
`touch state. In the context of the present description, a touch state refers to a predefined
`
`subset of the range of measurable contact pressures. For example, in one embodiment,
`
`Page 16 of 194
`
`
`
`
`
`
`
`
`-|7-
`
`contact pressure levels may be represented as one of four predefined touch states: no
`touch, light touch, medium touch, and heavy touch. As an option, the lowest touch state
`may require some form of proximity. In some embodiments, the user may be able to
`specify how the measurable range of contact pressures is partitioned across a predefined
`number of touch states. In one embodiment, the number and boundaries of touch states
`may be defined system-wide, for all applications. In another embodiment, the number
`
`and boundaries of touch states may be defined on a per-application basis.
`
`[00106] As shown, the selection area is calculated. See operation 406. In the context
`of the present description, the selection area refers to a region of the display, or a region
`of an object pictured on the display, which has been chosen for selection. The calculation
`of the selection area may be based upon one or more factors, including, but not limited to,
`a contact pressure level, a touch state, a contact area, a contact point, a selection
`boundary, a selection boundary geometry, and/or any other information derived from user
`interaction. In some embodiments, the selection area may be described as the pixels
`
`contained within a selection boundary.
`
`[00107] Inthe context of the present description, a selection boundary describes the
`boundary of an associated selection area, and is based upon a selection boundary
`geometry. Additionally, in the context of the present description, a selection boundary
`geometry refers to the shape of a selection boundary. In some embodiments, a selection
`boundary geometry may not have reference to actual size or location. For example, in
`one embodiment, a selection boundary geometry may be described using fractional
`
`coordinates on a unit square.
`
`[00108] In various embodiments, a selection area may be described, stored, and/or
`processed as the area contained within a selection boundary. A selection boundary may
`be represented by a selection boundary geometry, a screen location, and one or more
`transformations, such as scale or orientation. In one embodiment, the screen location is
`
`the point on which the transformed selection boundary geometry is centered.
`
`[00109] In some embodiments, the selection area may be described, stored, and/or
`
`processed, as a collection of cells that are part of an array. In one embodiment, this array
`
`Page 17 of 194
`
`
`
`
`
`
`
`
`-18-
`
`may directly correspond to pixels of a display. In another embodiment, the array may
`correspond to a grid based on a coordinate system specific to an object being displayed.
`In still another embodiment, the selection area may be described, stored, and processed as
`
`a mathematical function that defines the boundary of the selection area.
`
`[00110] In one embodiment, the shape of a selection boundary may be one of a
`plurality of selection boundary geometries predefined within an operating system. In
`another embodiment, the shape of a selection boundary may be predefined within an
`application, independent of selection boundary geometries defined within the operating
`system. In yet another embodiment, the selection boundary geometry may be specified
`by auser. In still another embodiment, the geometry of the selection boundary may
`depend upon at least one form of user input, including but not limited to contact pressure,
`number of fingers in contact with the screen, device orientation, location of user fingers,
`and/or any other form of user input. Finally, in another embodiment, the geometry of a
`selection boundary may depend upon the content within or near a location associated
`with the selection (e.g. contact point, finger location, focal point of user’s eyes, cursor
`
`location, etc.).
`
`[00111] In some embodiments, selection boundaries may have geometries based upon
`simple shapes, including, but not limited to, ovals, circles, triangles, squares, rectangles,
`and/or higher order polygons. In other embodiments, selection boundaries may be based
`upon regular shapes (e.g. a star, a plus sign, etc.). In one embodiment, a selection
`
`boundary may be based upon the geometry of a contact area.
`
`[00112] There are numerous ways in which a selection boundary may be described,
`stored, and/or processed. In various embodiments, a selection boundary may be
`represented by the combination of an established selection boundary geometry with a
`
`location, a scale, and/or a rotation.
`
`[00113] In one embodiment, a selection boundary may be described using the vertex
`coordinates of a selection boundary geometry, in conjunction with location, scale, and/or
`
`rotation factors. For example, a rectangular selection boundary geometry may be
`
`Page 18 of 194
`
`
`
`
`
`
`
`
`-19-
`
`described by the coordinates of the four corners. As an option, the vertices of a selection
`
`boundary geometry may be described using coordinates within the unit square.
`
`[00114] In another embodiment, a selection boundary may be described using a
`mathematical function representing an established selection boundary geometry, in
`conjunction with location, scale, and/or rotation factors. For example, an elliptical
`selection boundary geometry may be described by an ellipse function whose foci depend
`
`upon the scale factor.
`
`[00115] In yet another embodiment, a selection boundary may be described using a
`spline-based representation of a selection boundary geometry, in conjunction with
`location, scale, and/or rotation factors. For example, a curled selection boundary
`geometry may be described using splines combined with location, scale, and rotation
`
`factors.
`
`[00116] In other embodiments, a selection boundary may be described by a bitmap
`representation of a selection boundary geometry, in conjunction with location, scale,
`and/or rotation factors. For example, in one embodiment, a bitmap representation of a
`contact area may be used as a selection boundary geometry associated with a selection
`boundary. In another embodiment, a bitmap representation of an irregular shape may be
`used as a selection boundary geometry associated with a selection boundary. In this way,
`geometries which