SikuliX

[1.1.0-Beta1] click() fails on anything but Screen(0) - fixed in Beta2

Bug #1332260 reported by Tak Eda
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
SikuliX
Fix Committed
Critical
RaiMan
SikuliX 1.1.0 "SikuliX"

Bug Description

SikuliX Setup Build: 1.1.0-Beta1 2014-05-30-18

My sikuli script is able to locate the button fine and highlight even highlights the correct button, however click fails

Thinking that click was failing due to color change, I explicitly grabbed the location (coordinates) from the match and passed it to click but click still fails

I have tried to put in wait(), set ClickDelay, used hover before clicking: nothing works.

My script fails if it runs on anything but Screen(0)

If I keep the screen I'm trying to click on on my primary display, the script executes just fine!

The issue is that the script outputs all the correct coordinates as per Screen(1) but the click actually happens on Screen(0)!

Here is what my code looks like:

Settings.setShowActions(True)
Settings.ActionLogs = True
Settings.InfoLogs = True
Settings.DebugLogs = True

SCR1 = Screen(1)
p1 = Pattern("1402973791825.png")
p2 = Pattern("1402972955014.png")

goOn = True

while goOn:
    try:
        x1 = SCR1.find(p1.exact())
        print x1
        #hover(x1)
        goOn = False
    except FindFailed:
        x2 = SCR1.find(p2.exact())
        x2l = x2.getTarget()
        print x2l
        x2h = x2.hover()
        print x2h
        x2h = x2.hover()
        print x2h

        x2.highlight(5)
        # icon on hover becomes darker

        #sleep(100)
        x2c = x2.doubleClick()
        print x2c
        x2.highlight(5)
        x2.hover(p2)
        wait(0.5)
        Settings.ClickDelay = 0.3
        x2c = x2.click(p2)
        print x2c
        goOn = False
        x2c = x2.mouseDown(Button.LEFT)
        print x2c
        x2c = x2.mouseUp(Button.LEFT)
        print x2c

Revision history for this message
RaiMan (raimund-hocke) wrote : Re: [1.1.0-Beta1] click() fails on anything but Screen(0)

I am currently working hard on a solution for this problem - nearly fixed ;-)

summary: - click() fails on anything but Screen(0)
+ [1.1.0-Beta1] click() fails on anything but Screen(0)
Changed in sikuli:
status: New → Fix Committed
importance: Undecided → High
assignee: nobody → RaiMan (raimund-hocke)
milestone: none → 1.1.0
Revision history for this message
Tak Eda (takeda) wrote :

@RaiMan, Thanks again for looking into this.

I can definitely help you troubleshoot and (perhaps) FIX this!

The info messages actually show the correct coordinates where the click SHOULD go but the actual click happens on the other screen

Tell me where to look and how to turn more tracing on for the actual click function and I can work on this with you.

It seems as if Sikuli is correctly computing the coordinates, but the actual (AWT?) library is not translating it to the right onscreen coordinates?

Revision history for this message
RaiMan (raimund-hocke) wrote :

Thanks for wanting to help.

But I think I have fixed it now (tested on Mac and Windows).

A new build will be available on Monday (the git repo is up to date).

Revision history for this message
Tak Eda (takeda) wrote :

@RaiMan, that was quick!

Thanks again.

Can you point me to the commit on github so I can see what the fix was?

I'll anxiously wait for the Monday release.

Have a great weekend!

Revision history for this message
RaiMan (raimund-hocke) wrote :

--- Can you point me to the commit on github
not only one commit, nearly every commit since this one
https://github.com/RaiMan/SikuliX-2014/commit/d5b48fd296d4ba63a5c8894d70f14fb5eb2f66f1
has something to do with the complete (again :-( revision of the features where the use of AWT Robot was involved.
- Screen.initScreens()
- RobotDesktop
- OverlayCapturePrompt
- ScreenUnion
The main design flaw was the idea, to always have the screen with (0,0) as Screen(0) and hence primary screen.
Now I leave the screens in the order as they are reported by the underlying graphics system, which might lead to the case, where the primary screen (Mac: the screen having the menu bar, Windows: the screen having the desktop, Linux: I cannot check) now is Screen(n), where n > 0.
Mouse handling now is done using an unbound robot instance, which seems to work without the need to do any point translation.

There is one problem left: on Mac 10.9, the overlay capture prompt is not shown on the primary screen in multi monitor situation.

Revision history for this message
Tak Eda (takeda) wrote :

@RaiMan, Thanks again.

Has the JAR file on the download file been updated with the fix (or is there a location I can just pickup the compiled jar without having to compile it myself)?

Revision history for this message
RaiMan (raimund-hocke) wrote :

The new build will be available during the next 12 hours.

Revision history for this message
Tak Eda (takeda) wrote :

@RaiMan, will running the setup grab the latest files with this fix automatically or do I need to download the updated jars manually for my script to work with multiple monitors?

Revision history for this message
RaiMan (raimund-hocke) wrote :

Sorry, but I did not upload a new version yet.

But I will do that tomorrow.

In any case, you have to download the new version of sikulixsetup.jar (preferably in an empty folder) and run setup again, to get the latest fixes.
Beginning next week, I will offer nightly builds of the development head, that work the same way.

An inline update will only be available beginning with version 1.2 somewhen beginning of next year (or earlier with the betas and nightly builds)

Sorry, that you have to wait.

RaiMan (raimund-hocke)
Changed in sikuli:
importance: High → Critical
summary: - [1.1.0-Beta1] click() fails on anything but Screen(0)
+ [1.1.0-Beta1] click() fails on anything but Screen(0) - fixed in Beta2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.