Go programming - Learning HTTP requestsmultiple threaded program vb.netParsing a page's source code for URLsBouncing ball programLong running, frequent async advice?Multithreading pthread programIs this program susceptible to a buffer overflow exploit?cat program in GoGolang Tour Web Crawler ExerciseDiff for successive sorted sliceCLI Utility Go - First Go Program

Why does a 97 / 92 key piano exist by Bösendorfer?

Air travel with refrigerated insulin

Can I run 125khz RF circuit on a breadboard?

Confusion over Hunter with Crossbow Expert and Giant Killer

Deciphering cause of death?

Personal or impersonal in a technical resume

What should be the ideal length of sentences in a blog post for ease of reading?

Difference between shutdown options

Is there a reason to prefer HFS+ over APFS for disk images in High Sierra and/or Mojave?

How do I fix the group tension caused by my character stealing and possibly killing without provocation?

Pre-Employment Background Check With Consent For Future Checks

Origin of pigs as a species

What is the meaning of "You've never met a graph you didn't like?"

Why is participating in the European Parliamentary elections used as a threat?

Make a Bowl of Alphabet Soup

How to preserve electronics (computers, iPads and phones) for hundreds of years

Telemetry for feature health

Check if object is null and return null

Should I warn a new PhD Student?

Giving feedback to someone without sounding prejudiced

How to make money from a browser who sees 5 seconds into the future of any web page?

What in this world is she trying to say?

Cumulative Sum using Java 8 stream API

Language involving irrational number is not a CFL



Go programming - Learning HTTP requests


multiple threaded program vb.netParsing a page's source code for URLsBouncing ball programLong running, frequent async advice?Multithreading pthread programIs this program susceptible to a buffer overflow exploit?cat program in GoGolang Tour Web Crawler ExerciseDiff for successive sorted sliceCLI Utility Go - First Go Program













0












$begingroup$


I'm beginning to learn about Golang and I would like to have some advice about the following program.



package main

import (
"fmt"
"net/http"
"time"
)

const BenchmarkTry = 1000

type PageBenchmark struct
url string
time int64 // microseconds


func execBenchmark(url string, channel chan PageBenchmark)
totalExecTimeChan := make(chan int64, BenchmarkTry) // set size to prevent blocked goroutine
totalExecTime := int64(0)

// start all the goroutines
for i := 0; i < BenchmarkTry; i++
go execHttpRequest(url, totalExecTimeChan)


// catch new values from totalExecTimeChan values come from execHttpRequest()) and add it to the total
for i := 0; i < BenchmarkTry; i++
totalExecTime += <-totalExecTimeChan // waiting to get a value from one of the goroutines started in the previous for loop


channel <- PageBenchmarkurl, totalExecTime / BenchmarkTry


// exec http request and attach exec time to channel
func execHttpRequest(url string, channel chan int64)
begin := time.Now()
_, _ = http.Get(url)
channel <- time.Since(begin).Nanoseconds() / 1000000 // convert to milliseconds


func main()
sites := [...]string



pages := [...]string



benchmarkChan := make(chan PageBenchmark, len(sites)*len(pages)) // set size to prevent blocked goroutine

begin := time.Now()
fmt.Println("Beginning !")

// start all the goroutines
for site := range sites
for page := range pages
go execBenchmark(sites[site]+pages[page], benchmarkChan)



// catch new values from benchmarkChan and "print" the PageBenchmark
for i := 0; i < len(sites)*len(pages); i++
benchmark := <-benchmarkChan
fmt.Printf("Url : %vnResponse Time : %d msnn", benchmark.url, benchmark.time)


// print execution time
fmt.Println("End.")
fmt.Println(fmt.Sprintf("%d ms", time.Since(begin).Nanoseconds()/1000000))




So basically here, I'm making HTTP requests (GET method) to multiple urls on multiple web sites. 1000 requests by url in this example.



For now, I just want to start some goroutines in this order :



  • (First goroutine) mainroutine : start a benchmark (goroutine) foreach web site pages, get the average execution time and print it


  • (Second goroutine) page routine : start 1000 goroutines (benchmark tries), get the execution times from a channel and store the average execution time in an other channel


  • (Third goroutine) Execute an HTTP request on the page and store the execution time in a channel


Actually, this piece of code works but I'm not sure that it works as intended.



  • Is that the case ?


  • Is the execution scheduling valid ?


  • Is it appropriate to use defined size channels here ?


  • Is there a better/more effective way to achieve this task ?


I would be very grateful if you would please take some time to read this topic. Thank you and have a nice day.










share|improve this question









New contributor




hunomina is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$
















    0












    $begingroup$


    I'm beginning to learn about Golang and I would like to have some advice about the following program.



    package main

    import (
    "fmt"
    "net/http"
    "time"
    )

    const BenchmarkTry = 1000

    type PageBenchmark struct
    url string
    time int64 // microseconds


    func execBenchmark(url string, channel chan PageBenchmark)
    totalExecTimeChan := make(chan int64, BenchmarkTry) // set size to prevent blocked goroutine
    totalExecTime := int64(0)

    // start all the goroutines
    for i := 0; i < BenchmarkTry; i++
    go execHttpRequest(url, totalExecTimeChan)


    // catch new values from totalExecTimeChan values come from execHttpRequest()) and add it to the total
    for i := 0; i < BenchmarkTry; i++
    totalExecTime += <-totalExecTimeChan // waiting to get a value from one of the goroutines started in the previous for loop


    channel <- PageBenchmarkurl, totalExecTime / BenchmarkTry


    // exec http request and attach exec time to channel
    func execHttpRequest(url string, channel chan int64)
    begin := time.Now()
    _, _ = http.Get(url)
    channel <- time.Since(begin).Nanoseconds() / 1000000 // convert to milliseconds


    func main()
    sites := [...]string



    pages := [...]string



    benchmarkChan := make(chan PageBenchmark, len(sites)*len(pages)) // set size to prevent blocked goroutine

    begin := time.Now()
    fmt.Println("Beginning !")

    // start all the goroutines
    for site := range sites
    for page := range pages
    go execBenchmark(sites[site]+pages[page], benchmarkChan)



    // catch new values from benchmarkChan and "print" the PageBenchmark
    for i := 0; i < len(sites)*len(pages); i++
    benchmark := <-benchmarkChan
    fmt.Printf("Url : %vnResponse Time : %d msnn", benchmark.url, benchmark.time)


    // print execution time
    fmt.Println("End.")
    fmt.Println(fmt.Sprintf("%d ms", time.Since(begin).Nanoseconds()/1000000))




    So basically here, I'm making HTTP requests (GET method) to multiple urls on multiple web sites. 1000 requests by url in this example.



    For now, I just want to start some goroutines in this order :



    • (First goroutine) mainroutine : start a benchmark (goroutine) foreach web site pages, get the average execution time and print it


    • (Second goroutine) page routine : start 1000 goroutines (benchmark tries), get the execution times from a channel and store the average execution time in an other channel


    • (Third goroutine) Execute an HTTP request on the page and store the execution time in a channel


    Actually, this piece of code works but I'm not sure that it works as intended.



    • Is that the case ?


    • Is the execution scheduling valid ?


    • Is it appropriate to use defined size channels here ?


    • Is there a better/more effective way to achieve this task ?


    I would be very grateful if you would please take some time to read this topic. Thank you and have a nice day.










    share|improve this question









    New contributor




    hunomina is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.







    $endgroup$














      0












      0








      0





      $begingroup$


      I'm beginning to learn about Golang and I would like to have some advice about the following program.



      package main

      import (
      "fmt"
      "net/http"
      "time"
      )

      const BenchmarkTry = 1000

      type PageBenchmark struct
      url string
      time int64 // microseconds


      func execBenchmark(url string, channel chan PageBenchmark)
      totalExecTimeChan := make(chan int64, BenchmarkTry) // set size to prevent blocked goroutine
      totalExecTime := int64(0)

      // start all the goroutines
      for i := 0; i < BenchmarkTry; i++
      go execHttpRequest(url, totalExecTimeChan)


      // catch new values from totalExecTimeChan values come from execHttpRequest()) and add it to the total
      for i := 0; i < BenchmarkTry; i++
      totalExecTime += <-totalExecTimeChan // waiting to get a value from one of the goroutines started in the previous for loop


      channel <- PageBenchmarkurl, totalExecTime / BenchmarkTry


      // exec http request and attach exec time to channel
      func execHttpRequest(url string, channel chan int64)
      begin := time.Now()
      _, _ = http.Get(url)
      channel <- time.Since(begin).Nanoseconds() / 1000000 // convert to milliseconds


      func main()
      sites := [...]string



      pages := [...]string



      benchmarkChan := make(chan PageBenchmark, len(sites)*len(pages)) // set size to prevent blocked goroutine

      begin := time.Now()
      fmt.Println("Beginning !")

      // start all the goroutines
      for site := range sites
      for page := range pages
      go execBenchmark(sites[site]+pages[page], benchmarkChan)



      // catch new values from benchmarkChan and "print" the PageBenchmark
      for i := 0; i < len(sites)*len(pages); i++
      benchmark := <-benchmarkChan
      fmt.Printf("Url : %vnResponse Time : %d msnn", benchmark.url, benchmark.time)


      // print execution time
      fmt.Println("End.")
      fmt.Println(fmt.Sprintf("%d ms", time.Since(begin).Nanoseconds()/1000000))




      So basically here, I'm making HTTP requests (GET method) to multiple urls on multiple web sites. 1000 requests by url in this example.



      For now, I just want to start some goroutines in this order :



      • (First goroutine) mainroutine : start a benchmark (goroutine) foreach web site pages, get the average execution time and print it


      • (Second goroutine) page routine : start 1000 goroutines (benchmark tries), get the execution times from a channel and store the average execution time in an other channel


      • (Third goroutine) Execute an HTTP request on the page and store the execution time in a channel


      Actually, this piece of code works but I'm not sure that it works as intended.



      • Is that the case ?


      • Is the execution scheduling valid ?


      • Is it appropriate to use defined size channels here ?


      • Is there a better/more effective way to achieve this task ?


      I would be very grateful if you would please take some time to read this topic. Thank you and have a nice day.










      share|improve this question









      New contributor




      hunomina is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.







      $endgroup$




      I'm beginning to learn about Golang and I would like to have some advice about the following program.



      package main

      import (
      "fmt"
      "net/http"
      "time"
      )

      const BenchmarkTry = 1000

      type PageBenchmark struct
      url string
      time int64 // microseconds


      func execBenchmark(url string, channel chan PageBenchmark)
      totalExecTimeChan := make(chan int64, BenchmarkTry) // set size to prevent blocked goroutine
      totalExecTime := int64(0)

      // start all the goroutines
      for i := 0; i < BenchmarkTry; i++
      go execHttpRequest(url, totalExecTimeChan)


      // catch new values from totalExecTimeChan values come from execHttpRequest()) and add it to the total
      for i := 0; i < BenchmarkTry; i++
      totalExecTime += <-totalExecTimeChan // waiting to get a value from one of the goroutines started in the previous for loop


      channel <- PageBenchmarkurl, totalExecTime / BenchmarkTry


      // exec http request and attach exec time to channel
      func execHttpRequest(url string, channel chan int64)
      begin := time.Now()
      _, _ = http.Get(url)
      channel <- time.Since(begin).Nanoseconds() / 1000000 // convert to milliseconds


      func main()
      sites := [...]string



      pages := [...]string



      benchmarkChan := make(chan PageBenchmark, len(sites)*len(pages)) // set size to prevent blocked goroutine

      begin := time.Now()
      fmt.Println("Beginning !")

      // start all the goroutines
      for site := range sites
      for page := range pages
      go execBenchmark(sites[site]+pages[page], benchmarkChan)



      // catch new values from benchmarkChan and "print" the PageBenchmark
      for i := 0; i < len(sites)*len(pages); i++
      benchmark := <-benchmarkChan
      fmt.Printf("Url : %vnResponse Time : %d msnn", benchmark.url, benchmark.time)


      // print execution time
      fmt.Println("End.")
      fmt.Println(fmt.Sprintf("%d ms", time.Since(begin).Nanoseconds()/1000000))




      So basically here, I'm making HTTP requests (GET method) to multiple urls on multiple web sites. 1000 requests by url in this example.



      For now, I just want to start some goroutines in this order :



      • (First goroutine) mainroutine : start a benchmark (goroutine) foreach web site pages, get the average execution time and print it


      • (Second goroutine) page routine : start 1000 goroutines (benchmark tries), get the execution times from a channel and store the average execution time in an other channel


      • (Third goroutine) Execute an HTTP request on the page and store the execution time in a channel


      Actually, this piece of code works but I'm not sure that it works as intended.



      • Is that the case ?


      • Is the execution scheduling valid ?


      • Is it appropriate to use defined size channels here ?


      • Is there a better/more effective way to achieve this task ?


      I would be very grateful if you would please take some time to read this topic. Thank you and have a nice day.







      multithreading go http benchmarking






      share|improve this question









      New contributor




      hunomina is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|improve this question









      New contributor




      hunomina is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|improve this question




      share|improve this question








      edited 7 mins ago









      Alex

      563210




      563210






      New contributor




      hunomina is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked 1 hour ago









      hunominahunomina

      1




      1




      New contributor




      hunomina is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      hunomina is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      hunomina is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.




















          0






          active

          oldest

          votes











          Your Answer





          StackExchange.ifUsing("editor", function ()
          return StackExchange.using("mathjaxEditing", function ()
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
          );
          );
          , "mathjax-editing");

          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "196"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );






          hunomina is a new contributor. Be nice, and check out our Code of Conduct.









          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f215883%2fgo-programming-learning-http-requests%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          hunomina is a new contributor. Be nice, and check out our Code of Conduct.









          draft saved

          draft discarded


















          hunomina is a new contributor. Be nice, and check out our Code of Conduct.












          hunomina is a new contributor. Be nice, and check out our Code of Conduct.











          hunomina is a new contributor. Be nice, and check out our Code of Conduct.














          Thanks for contributing an answer to Code Review Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f215883%2fgo-programming-learning-http-requests%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          名間水力發電廠 目录 沿革 設施 鄰近設施 註釋 外部連結 导航菜单23°50′10″N 120°42′41″E / 23.83611°N 120.71139°E / 23.83611; 120.7113923°50′10″N 120°42′41″E / 23.83611°N 120.71139°E / 23.83611; 120.71139計畫概要原始内容臺灣第一座BOT 模式開發的水力發電廠-名間水力電廠名間水力發電廠 水利署首件BOT案原始内容《小檔案》名間電廠 首座BOT水力發電廠原始内容名間電廠BOT - 經濟部水利署中區水資源局

          Prove that NP is closed under karp reduction?Space(n) not closed under Karp reductions - what about NTime(n)?Class P is closed under rotation?Prove or disprove that $NL$ is closed under polynomial many-one reductions$mathbfNC_2$ is closed under log-space reductionOn Karp reductionwhen can I know if a class (complexity) is closed under reduction (cook/karp)Check if class $PSPACE$ is closed under polyonomially space reductionIs NPSPACE also closed under polynomial-time reduction and under log-space reduction?Prove PSPACE is closed under complement?Prove PSPACE is closed under union?

          Is my guitar’s action too high? Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern)Strings too stiff on a recently purchased acoustic guitar | Cort AD880CEIs the action of my guitar really high?Μy little finger is too weak to play guitarWith guitar, how long should I give my fingers to strengthen / callous?When playing a fret the guitar sounds mutedPlaying (Barre) chords up the guitar neckI think my guitar strings are wound too tight and I can't play barre chordsF barre chord on an SG guitarHow to find to the right strings of a barre chord by feel?High action on higher fret on my steel acoustic guitar