* git-filter-repo For Windows (possibly a request for a bash for Windows primer) @ 2020-04-20 14:03 Kerry, Richard 2020-04-20 16:53 ` Elijah Newren 0 siblings, 1 reply; 6+ messages in thread From: Kerry, Richard @ 2020-04-20 14:03 UTC (permalink / raw) To: git@vger.kernel.org This follows my earlier query for which the answer was "use git-filter-repo". I've cloned git-filter-repo from GitHub, and I've installed the latest Python (version 3.8). I'm working on Windows. Windows 10 if it makes any difference, so git is Git For Windows. The first instruction for git-filter-repo is to copy it into the "--exec-path" of git. That fails, saying Access is Denied. So leaving aside the option to change the permissions I've followed the second half of the sentence and added my git-filter-repo location to my Path (or PATH if it's in bash). Now running "get filter-repo" gives: /usr/bin/env: 'python3.exe': No such file or directory My python executable is called python3.exe. There is a note in the git-filter-repo project about certain Windows installations where it isn't called python3, but that now raises in my mind the question of whether I need to tweak git-filter-repo to look for "python3.exe" rather than just "python3". Or can I take it that the Windows version of bash understands that it needs to add ".exe" automatically? Assuming that is handled automatically, what do I now need to do to satisfy the error message? Do I need to put something in bash-for-Windows' /usr/bin/env directory? Or is there something else entirely that I should be doing? And is it OK to ask about this on this list, or should I be going to a git-filter-branch resource? Or for Git For Windows? Regards, Richard. Richard Kerry BNCS Engineer, SI SOL Telco & Media Vertical Practice M: +44 (0)7812 325518 2nd Floor, MidCity Place, 71 High Holborn, London, WC1V 6EA https://webmail.siemens-it-solutions.com/owa/redir.aspx?C=9fb20d019e3e4cb99344d708709a3177&URL=mailto%3arichard.kerry%40atos.net This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git-filter-repo For Windows (possibly a request for a bash for Windows primer) 2020-04-20 14:03 git-filter-repo For Windows (possibly a request for a bash for Windows primer) Kerry, Richard @ 2020-04-20 16:53 ` Elijah Newren 2020-04-20 17:24 ` Elijah Newren 0 siblings, 1 reply; 6+ messages in thread From: Elijah Newren @ 2020-04-20 16:53 UTC (permalink / raw) To: Kerry, Richard; +Cc: git@vger.kernel.org On Mon, Apr 20, 2020 at 8:11 AM Kerry, Richard <richard.kerry@atos.net> wrote: > > > This follows my earlier query for which the answer was "use git-filter-repo". > I've cloned git-filter-repo from GitHub, and I've installed the latest Python (version 3.8). I'm working on Windows. Windows 10 if it makes any difference, so git is Git For Windows. > The first instruction for git-filter-repo is to copy it into the "--exec-path" of git. That fails, saying Access is Denied. So leaving aside the option to change the permissions I've followed the second half of the sentence and added my git-filter-repo location to my Path (or PATH if it's in bash). Now running "get filter-repo" gives: > /usr/bin/env: 'python3.exe': No such file or directory > > My python executable is called python3.exe. There is a note in the git-filter-repo project about certain Windows installations where it isn't called python3, but that now raises in my mind the question of whether I need to tweak git-filter-repo to look for "python3.exe" rather than just "python3". Or can I take it that the Windows version of bash understands that it needs to add ".exe" automatically? > > Assuming that is handled automatically, what do I now need to do to satisfy the error message? Do I need to put something in bash-for-Windows' /usr/bin/env directory? Or is there something else entirely that I should be doing? > > And is it OK to ask about this on this list, or should I be going to a git-filter-branch resource? Or for Git For Windows? > > Regards, > Richard. Emailing this list is fine. You can also open an issue on GitHub for the project if you prefer; I respond in both places. I may have limited ability to help with Windows issues since I don't have a Windows system to use or test on, though I have succeeded in fixing a few Windows bugs based on reporter feedback. (So maybe this list is better because more people can chime in with Windows experience?) I've got at least a dozen or so reports of people using it successfully on Windows so someone has figured it out. I'll try to provide a few pointers that might help. You don't have to use "#!/usr/bin/env python3" or "#!/usr/bin/env python". I could have put "#!/usr/bin/python3" there, but that might make it specific to my system. I figured other systems installed python3 somewhere other than /usr/bin/, but that /usr/bin/env should be somewhat common and should find the python installation. If /usr/bin/env doesn't happen to exist on your system, though, it is perfectly fine to have "#!/System/Commands/python3" or whatever is needed. If installation is difficult, you could consider using a package manager. In particular, the scoop package manager, built for Windows, was one of the first out there to package git-filter-repo -- it was even ahead of most the linux package managers. (And, yes, scoop can also be used to install python3 as far as I understand.) I haven't used it myself (because I don't have a Windows system as I mentioned before), but since they were so interested in packaging git-filter-repo and making "scoop install git-filter-repo" part of the instructions (https://github.com/newren/git-filter-repo/issues/20), they look like a friendly bunch that are willing to help. I also looked through the issues and PRs related to Windows, and thought that the following links particularly had interesting comments that might be useful to someone running on Windows: * https://github.com/newren/git-filter-repo/issues/36 * https://github.com/newren/git-filter-repo/pull/10 * https://github.com/newren/git-filter-repo/issues/48 I don't know if any of them will solve your particular problem, but they seemed more likely to be of interest or use than most. Also, I'll repeat the link for search for reports of Windows issues with git-filter-repo in case I missed one that might be useful to you: * https://github.com/newren/git-filter-repo/issues?q=is%3Aissue+windows Anyway, I hope at least one of those pointers is useful to you. Elijah ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git-filter-repo For Windows (possibly a request for a bash for Windows primer) 2020-04-20 16:53 ` Elijah Newren @ 2020-04-20 17:24 ` Elijah Newren 2020-04-20 18:46 ` Philip Oakley 0 siblings, 1 reply; 6+ messages in thread From: Elijah Newren @ 2020-04-20 17:24 UTC (permalink / raw) To: Kerry, Richard; +Cc: git@vger.kernel.org On Mon, Apr 20, 2020 at 9:53 AM Elijah Newren <newren@gmail.com> wrote: > > On Mon, Apr 20, 2020 at 8:11 AM Kerry, Richard <richard.kerry@atos.net> wrote: > > > > > > This follows my earlier query for which the answer was "use git-filter-repo". > > I've cloned git-filter-repo from GitHub, and I've installed the latest Python (version 3.8). I'm working on Windows. Windows 10 if it makes any difference, so git is Git For Windows. > > The first instruction for git-filter-repo is to copy it into the "--exec-path" of git. That fails, saying Access is Denied. So leaving aside the option to change the permissions I've followed the second half of the sentence and added my git-filter-repo location to my Path (or PATH if it's in bash). Now running "get filter-repo" gives: > > /usr/bin/env: 'python3.exe': No such file or directory > > > > My python executable is called python3.exe. There is a note in the git-filter-repo project about certain Windows installations where it isn't called python3, but that now raises in my mind the question of whether I need to tweak git-filter-repo to look for "python3.exe" rather than just "python3". Or can I take it that the Windows version of bash understands that it needs to add ".exe" automatically? > > > > Assuming that is handled automatically, what do I now need to do to satisfy the error message? Do I need to put something in bash-for-Windows' /usr/bin/env directory? Or is there something else entirely that I should be doing? > > > > And is it OK to ask about this on this list, or should I be going to a git-filter-branch resource? Or for Git For Windows? > > > > Regards, > > Richard. > > Emailing this list is fine. You can also open an issue on GitHub for > the project if you prefer; I respond in both places. I may have > limited ability to help with Windows issues since I don't have a > Windows system to use or test on, though I have succeeded in fixing a > few Windows bugs based on reporter feedback. (So maybe this list is > better because more people can chime in with Windows experience?) > I've got at least a dozen or so reports of people using it > successfully on Windows so someone has figured it out. I'll try to > provide a few pointers that might help. > > You don't have to use "#!/usr/bin/env python3" or "#!/usr/bin/env > python". I could have put "#!/usr/bin/python3" there, but that might > make it specific to my system. I figured other systems installed > python3 somewhere other than /usr/bin/, but that /usr/bin/env should > be somewhat common and should find the python installation. If > /usr/bin/env doesn't happen to exist on your system, though, it is > perfectly fine to have "#!/System/Commands/python3" or whatever is > needed. > > If installation is difficult, you could consider using a package > manager. In particular, the scoop package manager, built for Windows, > was one of the first out there to package git-filter-repo -- it was > even ahead of most the linux package managers. (And, yes, scoop can > also be used to install python3 as far as I understand.) I haven't > used it myself (because I don't have a Windows system as I mentioned > before), but since they were so interested in packaging > git-filter-repo and making "scoop install git-filter-repo" part of the > instructions (https://github.com/newren/git-filter-repo/issues/20), > they look like a friendly bunch that are willing to help. > > I also looked through the issues and PRs related to Windows, and > thought that the following links particularly had interesting comments > that might be useful to someone running on Windows: > * https://github.com/newren/git-filter-repo/issues/36 > * https://github.com/newren/git-filter-repo/pull/10 > * https://github.com/newren/git-filter-repo/issues/48 > I don't know if any of them will solve your particular problem, but > they seemed more likely to be of interest or use than most. Also, > I'll repeat the link for search for reports of Windows issues with > git-filter-repo in case I missed one that might be useful to you: > * https://github.com/newren/git-filter-repo/issues?q=is%3Aissue+windows Oh, also https://github.com/newren/git-filter-repo/issues/56 which suggests PowerShell may mess things up (and in a different way than issue 36 showed). ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git-filter-repo For Windows (possibly a request for a bash for Windows primer) 2020-04-20 17:24 ` Elijah Newren @ 2020-04-20 18:46 ` Philip Oakley 2020-04-21 14:52 ` Kerry, Richard 0 siblings, 1 reply; 6+ messages in thread From: Philip Oakley @ 2020-04-20 18:46 UTC (permalink / raw) To: Elijah Newren, Kerry, Richard; +Cc: git@vger.kernel.org On 20/04/2020 18:24, Elijah Newren wrote: > On Mon, Apr 20, 2020 at 9:53 AM Elijah Newren <newren@gmail.com> wrote: >> On Mon, Apr 20, 2020 at 8:11 AM Kerry, Richard <richard.kerry@atos.net> wrote: >>> >>> This follows my earlier query for which the answer was "use git-filter-repo". >>> I've cloned git-filter-repo from GitHub, and I've installed the latest Python (version 3.8). I'm working on Windows. Windows 10 if it makes any difference, so git is Git For Windows. >>> The first instruction for git-filter-repo is to copy it into the "--exec-path" of git. That fails, saying Access is Denied. So leaving aside the option to change the permissions I've followed the second half of the sentence and added my git-filter-repo location to my Path (or PATH if it's in bash). Now running "get filter-repo" gives: >>> /usr/bin/env: 'python3.exe': No such file or directory >>> >>> My python executable is called python3.exe. There is a note in the git-filter-repo project about certain Windows installations where it isn't called python3, but that now raises in my mind the question of whether I need to tweak git-filter-repo to look for "python3.exe" rather than just "python3". Or can I take it that the Windows version of bash understands that it needs to add ".exe" automatically? >>> >>> Assuming that is handled automatically, what do I now need to do to satisfy the error message? Do I need to put something in bash-for-Windows' /usr/bin/env directory? Or is there something else entirely that I should be doing? >>> >>> And is it OK to ask about this on this list, or should I be going to a git-filter-branch resource? Or for Git For Windows? >>> >>> Regards, >>> Richard. >> Emailing this list is fine. You can also open an issue on GitHub for >> the project if you prefer; I respond in both places. I may have >> limited ability to help with Windows issues since I don't have a >> Windows system to use or test on, though I have succeeded in fixing a >> few Windows bugs based on reporter feedback. (So maybe this list is >> better because more people can chime in with Windows experience?) >> I've got at least a dozen or so reports of people using it >> successfully on Windows so someone has figured it out. I'll try to >> provide a few pointers that might help. >> >> You don't have to use "#!/usr/bin/env python3" or "#!/usr/bin/env >> python". I could have put "#!/usr/bin/python3" there, but that might >> make it specific to my system. I figured other systems installed >> python3 somewhere other than /usr/bin/, but that /usr/bin/env should >> be somewhat common and should find the python installation. If >> /usr/bin/env doesn't happen to exist on your system, though, it is >> perfectly fine to have "#!/System/Commands/python3" or whatever is >> needed. >> >> If installation is difficult, you could consider using a package >> manager. In particular, the scoop package manager, built for Windows, >> was one of the first out there to package git-filter-repo -- it was >> even ahead of most the linux package managers. (And, yes, scoop can >> also be used to install python3 as far as I understand.) I haven't >> used it myself (because I don't have a Windows system as I mentioned >> before), but since they were so interested in packaging >> git-filter-repo and making "scoop install git-filter-repo" part of the >> instructions (https://github.com/newren/git-filter-repo/issues/20), >> they look like a friendly bunch that are willing to help. >> >> I also looked through the issues and PRs related to Windows, and >> thought that the following links particularly had interesting comments >> that might be useful to someone running on Windows: >> * https://github.com/newren/git-filter-repo/issues/36 >> * https://github.com/newren/git-filter-repo/pull/10 >> * https://github.com/newren/git-filter-repo/issues/48 >> I don't know if any of them will solve your particular problem, but >> they seemed more likely to be of interest or use than most. Also, >> I'll repeat the link for search for reports of Windows issues with >> git-filter-repo in case I missed one that might be useful to you: >> * https://github.com/newren/git-filter-repo/issues?q=is%3Aissue+windows > Oh, also https://github.com/newren/git-filter-repo/issues/56 which > suggests PowerShell may mess things up (and in a different way than > issue 36 showed). A similar Python issue occurs on Windows, as reported at https://github.com/git-for-windows/git/issues/2594 with a reply by dscho reminding that: This issue is listed specifically in the Known Issues in our Release Notes: Some console programs, most notably non-MSYS2 Python, PHP, Node and OpenSSL, interact correctly with MinTTY only when called through winpty (e.g. the Python console needs to be started as winpty python instead of just python). The installer even suggests (on the page titled "Configuring the terminal emulator to use with Git Bash") to Use Windows' default console window [...] which works well with Win32 console programs such as interactive Python or node.js [...] Hope that helps. -- Philip ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git-filter-repo For Windows (possibly a request for a bash for Windows primer) 2020-04-20 18:46 ` Philip Oakley @ 2020-04-21 14:52 ` Kerry, Richard 2020-04-25 14:15 ` Johannes Schindelin 0 siblings, 1 reply; 6+ messages in thread From: Kerry, Richard @ 2020-04-21 14:52 UTC (permalink / raw) To: Philip Oakley, Elijah Newren; +Cc: git@vger.kernel.org Thanks for everyone's advice and assistance. I had a look at all the links provided and wasn't convinced any of them quite covered my requirement, but I have now made some progress using Scoop. I had hoped that all I would need to do would be: 1. Install latest Python for Windows (3.8). Install for All Users means it's in C:\Program Files\Python38, and that's in the Path. 2. Clone git-filter-repo. 3. Point Path at git-filter-repo. (Alternative was to copy it into "exec-path" folder under Git, but that was protected.) 4. Run it. That didn't work: /usr/bin/env: 'python3': No such file or directory. That string pointed me at the shebang line of git-filter-repo, so I set about trying anything that I thought might point at my Python installation. Nothing I tried worked. Some things, like "/bin/sh", did indicate that the right file was being found and executed, but gave errors that it was being executed as a shell script, which of course it isn't. Any attempt to point at the python exe seemed to say things along the lines of "git can find it but not understand it" (sorry I can't now copy in the real error message as I had to close the Bash shell in order to update GfW.) So I gave up on getting it to run using raw Windows - well not entirely raw Windows as it's going to be run from Bash-for-git-for-windows (or so I thought). Installed Scoop. Got Scoop to install git-filter-repo and python. At last that lets me run it and have it do something useful. Though actually as Scoop gives its instructions for being run from PowerShell I'm not in the end running from bash at all. First attempt at running it was using: git filter-repo --analyze --source=core_system_archive which responded "not a git repository" even though core_system_archive does contain a ".git". But then git filter-repo --analyze --source=core_system_archive/.git or git filter-repo --analyze --source=./core_system_archive/.git didn't work either, same error, although it looks to me like core_system_archive/.git is a git repo. So would I need some different syntax to make that work? (And I've since noticed that examples use space rather than '=' to separate keyword from param, so I've tried that, with the same failures) So next option was to "cd .\core_system_archive", then just git filter-repo --analyze Which has worked. Though it isn't clear what was wrong with the parameter I used for --source. How am I supposed to reference a source repo without cd-ing into it? So thanks everyone for a lot of hints, but for me, under Windows, only the Scoop package manager managed to get all the right settings for filter-repo to work. And only now, having been rather rushing to get this to work due to other priorities, have I seen the comment about using pip to install it, so maybe that would also have worked. Regards, Richard. Oh, and having also seen Philip's reply (below) I don't think I would have recognized it as relating to my issue. What is winpty? Should that have gone in the shebang line of git-filter-repo in some form? From: Philip Oakley <philipoakley@iee.email> with a reply by dscho reminding that: This issue is listed specifically in the Known Issues in our Release Notes: Some console programs, most notably non-MSYS2 Python, PHP, Node and OpenSSL, interact correctly with MinTTY only when called through winpty (e.g. the Python console needs to be started as winpty python instead of just python). The installer even suggests (on the page titled "Configuring the terminal emulator to use with Git Bash") to Use Windows' default console window [...] which works well with Win32 console programs such as interactive Python or node.js [...] Regards Richard Kerry BNCS Engineer, SI SOL Telco & Media Vertical Practice M: +44 (0)7812 325518 2nd Floor, MidCity Place, 71 High Holborn, London, WC1V 6EA richard.kerry@atos.net This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git-filter-repo For Windows (possibly a request for a bash for Windows primer) 2020-04-21 14:52 ` Kerry, Richard @ 2020-04-25 14:15 ` Johannes Schindelin 0 siblings, 0 replies; 6+ messages in thread From: Johannes Schindelin @ 2020-04-25 14:15 UTC (permalink / raw) To: Kerry, Richard; +Cc: Philip Oakley, Elijah Newren, git@vger.kernel.org [-- Attachment #1: Type: text/plain, Size: 5568 bytes --] Hi Richard, On Tue, 21 Apr 2020, Kerry, Richard wrote: > Thanks for everyone's advice and assistance. > I had a look at all the links provided and wasn't convinced any of them > quite covered my requirement, but I have now made some progress using > Scoop. > > I had hoped that all I would need to do would be: > 1. Install latest Python for Windows (3.8). Install for All Users means it's in C:\Program Files\Python38, and that's in the Path. I installed it via the Microsoft Store. This put it into my user-specific AppData, and it added two items to the PATH. Unfortunately, even calling `python3.exe` from the PATH does not work: it is apparently some placeholder that MSYS2 (and therefore Git Bash and `env.exe`) have no idea what to do with. What did work for me was `env python`. Of course, I don't have Python 2.x installed, so that might not be an option for you. > 2. Clone git-filter-repo. > 3. Point Path at git-filter-repo. (Alternative was to copy it into "exec-path" folder under Git, but that was protected.) For the record, Git for Windows also adds the `bin` folder in your home directory to the PATH. That is where you are supposed to put additional things. Git for Windows considers C:\Program Files\Git its own home, and it might remove your files without prior warning during an upgrade. It will leave `~/bin/` alone, of course. Hopefully these two hints prove useful to you. Ciao, Johannes > 4. Run it. > > That didn't work: /usr/bin/env: 'python3': No such file or directory. > That string pointed me at the shebang line of git-filter-repo, so I set about trying anything that I thought might point at my Python installation. Nothing I tried worked. Some things, like "/bin/sh", did indicate that the right file was being found and executed, but gave errors that it was being executed as a shell script, which of course it isn't. Any attempt to point at the python exe seemed to say things along the lines of "git can find it but not understand it" (sorry I can't now copy in the real error message as I had to close the Bash shell in order to update GfW.) > > So I gave up on getting it to run using raw Windows - well not entirely raw Windows as it's going to be run from Bash-for-git-for-windows (or so I thought). > Installed Scoop. Got Scoop to install git-filter-repo and python. At last that lets me run it and have it do something useful. > Though actually as Scoop gives its instructions for being run from PowerShell I'm not in the end running from bash at all. > > First attempt at running it was using: > > git filter-repo --analyze --source=core_system_archive > which responded "not a git repository" even though core_system_archive does contain a ".git". > But then > git filter-repo --analyze --source=core_system_archive/.git > or > git filter-repo --analyze --source=./core_system_archive/.git > didn't work either, same error, although it looks to me like core_system_archive/.git is a git repo. So would I need some different syntax to make that work? (And I've since noticed that examples use space rather than '=' to separate keyword from param, so I've tried that, with the same failures) > So next option was to "cd .\core_system_archive", then just > git filter-repo --analyze > Which has worked. > Though it isn't clear what was wrong with the parameter I used for --source. How am I supposed to reference a source repo without cd-ing into it? > > So thanks everyone for a lot of hints, but for me, under Windows, only the Scoop package manager managed to get all the right settings for filter-repo to work. > And only now, having been rather rushing to get this to work due to other priorities, have I seen the comment about using pip to install it, so maybe that would also have worked. > > Regards, > Richard. > > Oh, and having also seen Philip's reply (below) I don't think I would have recognized it as relating to my issue. > What is winpty? Should that have gone in the shebang line of git-filter-repo in some form? > > > From: Philip Oakley <philipoakley@iee.email> > > with a reply by dscho reminding that: > > This issue is listed specifically in the Known Issues in our Release Notes: > > Some console programs, most notably non-MSYS2 Python, PHP, Node > and OpenSSL, interact correctly with MinTTY only when called through > winpty (e.g. the Python console needs to be started as winpty python > instead of just python). > > The installer even suggests (on the page titled "Configuring the > terminal emulator to use with Git Bash") to > Use Windows' default console window > [...] which works well with Win32 console programs such as > interactive Python or node.js [...] > > > Regards > Richard Kerry > > BNCS Engineer, SI SOL Telco & Media Vertical Practice > M: +44 (0)7812 325518 > 2nd Floor, MidCity Place, 71 High Holborn, London, WC1V 6EA > richard.kerry@atos.net > > This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy > it. As its integrity cannot be secured on the Internet, the Atos group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free > and will not be liable for any damages resulting from any virus transmitted. > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-04-25 14:15 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-04-20 14:03 git-filter-repo For Windows (possibly a request for a bash for Windows primer) Kerry, Richard 2020-04-20 16:53 ` Elijah Newren 2020-04-20 17:24 ` Elijah Newren 2020-04-20 18:46 ` Philip Oakley 2020-04-21 14:52 ` Kerry, Richard 2020-04-25 14:15 ` Johannes Schindelin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).